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Abstract 


This  research  develops  an  analytical  technique  to  estimate  the  impact  of  funding 
curtailment  on  an  R&D  program.  The  method  quickly  produces  a  revised  budget  by  year 
for  an  on-going  R&D  program  when  funding  in  one  year  is  reduced.  We  assume 
program  requirements  remain  unchanged.  The  program  duration  may  be  unchanged  or 
“stretched”  to  a  later  completion  date.  We  use  the  Rayleigh  and  Weibull  functions  to 
model  expenditure  profiles,  which  forms  basis  of  the  analytical  approach.  Our  proposed 
methodology  accounts  for  budget  outlay  rates  and  inflation.  We  validate  the  proposed 
analytical  technique  using  historical  cost  data  from  several  programs. 


REVISING  R&D  PROGRAM  BUDGETS  WHEN  CONSIDERING 


FUNDING  CURTAILMENT  WITH  A  WEIBULL  MODEL 


I.  Introduction  to  the  Research 


General  Issue 

The  military  departments  and  the  Office  of  the  Secretary  of  Defense  (OSD) 
conduct  annual  reviews  of  their  programs  and  budgets.  At  times,  the  total  program 
requests  exceed  prescribed  funding  goals.  Rather  than  terminate  programs,  funding 
levels  for  development  and  production  programs  are  often  curtailed  to  comply  with 
spending  limitations.  The  funding  reduction  lessens  the  effort  on  the  project  in  that  fiscal 
year,  and  may  extend  the  completion  date  if  the  program  requirements  remain  unchanged. 

For  programs  in  the  production  phase,  budget  analysts  can  easily  determine  and 
revise  the  budget  profile.  Cost  progress  curves,  sometimes  called  learning  curves,  relate 
the  quantity  produced  in  any  fiscal  year  to  the  required  cost.  An  analyst  can  reduce 
annual  production  quantities  until  the  cost  is  below  funding  limits. 

For  programs  in  the  research  and  development  (R&D)  phase,  the  impact  of  a 
budget  reduction  is  less  certain.  Without  a  corresponding  reduction  in  program  content, 
the  imposed  funding  constraints  generally  prevent  acquisition  objectives  and  milestones 
to  be  reached  as  scheduled;  hence,  the  timeline  is  “stretched.” 

Besides  just  shifting  effort  in  time,  the  R&D  program  likely  encounters 
inefficiencies  that  increase  costs.  For  example,  development  of  a  component  may  be 
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stopped  as  part  of  reducing  the  effort,  but  an  additional  restart  cost  is  incurred  when  the 
development  is  resumed.  Another  example  is  a  group  of  engineers  may  not  be  as 
productive  because  the  required  interface  development  is  delayed.  Although  in  the  short¬ 
term  money  is  “saved,”  financial  analyst  realize  that  in  the  long  run  the  program  will 
incur  additional  cost  more  than  the  amount  “saved.” 

Putman  (1978:348)  and  Gallagher  and  Lee  (1996:52)  indicate  that  there  is  an 
efficient  rate  at  which  a  program  in  the  R&D  phase  expends  funds.  Jarvis  and  Pohl 
(1999)  review  three  models  for  expenditure  rates.  Lee,  Hogue,  and  Gallagher  (Lee, 
1997:3 1)  report  that  the  Rayleigh  Model  fit  contract  expenditures  from  20  defense  R&D 
acquisition  programs  and  how  to  account  for  outlay  rates. 

Once  money  is  removed  from  a  program,  the  funding  profile  for  the  program  is 
permanently  changed.  The  changed  profile  should  reflect  a  different  efficient 
expenditure  profile.  We  depict  the  initial  expenditure  and  revised  expenditure  profile 
with  a  Weibull  model. 

Specific  Problem 

Cost  progress  (learning)  curves  provide  quick  means  to  estimate  revised  budgets 
on  production  programs.  The  focus  of  this  research  is  to  develop  an  analytical  model  to 
estimate  the  impact  of  funding  curtailment  on  R&D  programs.  This  research  develops  an 
approach  to  assess  funding  curtailment  impact  to  an  R&D  program’s  overall  cost  and 
schedule. 
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Research  Benefits 


This  research  provides  the  military  financial  communities,  such  as  the  Deputy 
Undersecretary  of  the  Air  Force  for  Financial  Management  (SAF/FM)  and  OSD  Program 
Analysis  &  Evaluation  (PA&E),  with  a  method  to  develop  general  estimates  of  total  cost 
with  revised  budgets  when  they  consider  curtailment  of  funding  to  a  program  in  the  R&D 
phase. 

Other  Considerations 

The  purpose  of  this  research  is  to  provide  financial  organizations  in  the  military 
departments  and  OSD  PA&E  with  a  model  that  can  be  used  quickly  to  give  a  good 
estimate  of  the  effects  of  the  proposed  curtailment  of  funding.  The  usefulness  of  the 
proposed  model  is  the  speed  at  which  the  impact  of  the  curtailment  can  be  determined. 
This  model  is  not  a  substitute  for  the  program  office’s  detailed  estimates.  Our  approach 
provides  a  quick  approximation  while  leaving  the  specifics  to  the  program  office. 

We  also  answer  other  questions  related  to  this  topic.  When  funding  is  cut  to 
“save”  money  in  one  year,  can  we  quantify,  in  dollars,  the  impact  this  action  has  in  the 
total  cost  of  the  program?  How  much  does  that  “curtailed”  dollar  “cost”  in  relation  to  the 
program?  For  example,  does  curtailing  $200  million  today  add  $350  million  to  the  total 
cost  of  the  R&D  program?  What  is  the  resulting  required  budget  profile  to  support  an 
efficient  expenditure  plan?  Can  budget  analysts  use  the  Rayleigh  model  to  provide 
revised  budget  profiles  of  defense  acquisition  programs  in  the  research  and  development 
phase  that  have  had  their  funding  curtailed  in  one  fiscal  year? 
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This  chapter  introduces  the  general  problem  associated  with  funding  curtailment 
for  a  program  in  the  R&D  phase.  This  chapter  highlights  some  of  the  consequences 
associated  with  curtailing  program  funding.  Chapter  Two  reviews  the  literature  on  cost 
modeling  R&D  programs.  Chapter  Two  identifies  the  Rayleigh  Cost  Model  as  a  valuable 
tool  in  forecasting  budgets.  Chapter  Three  describes  the  analytical  approach  we  propose 
to  forecast  a  budget  when  funding  curtailment  is  proposed.  Chapter  Four  provides  the 
results  of  using  historical  programs  as  a  validation  of  the  methodology  outlined  in 
Chapter  Three.  Finally,  Chapter  Five  presents  general  conclusions  and  findings  on  our 
proposed  methodology. 
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II  Literature  Review  and  Concept  Definition 


Chapter  Overview 

Development  of  an  appropriate  funding  profile  for  an  R&D  program  is  a  key 
concern  for  those  with  a  vested  interest  in  the  success  of  the  program.  If  the  program  is 
not  sufficiently  funded,  advancement  of  the  program  is  hindered.  If  the  program  is  over- 
funded,  money  spent  on  the  program  could  be  used  elsewhere  with  little  impact  to  the 
program  itself.  Cost  analysts  employ  different  mathematical  modeling  techniques  to 
develop  the  appropriate  funding  profiles. 

This  chapter  first  presents  current  mathematical  models  detailed  in  the  literature. 
The  Rayleigh  Model,  the  final  model  presented,  is  explored  in  depth.  The  Rayleigh 
probability  density  function  and  cumulative  distribution  function  are  presented  and 
graphed.  The  discussion  includes  the  parameters  and  variables  necessary  for  the 
Rayleigh  Model  to  be  an  accurate  forecasting  tool.  This  chapter  concludes  with  an 
explanation  of  how  the  Rayleigh  model  applies  to  funding  issues. 

Large  defense  R&D  programs  are  complex  and  take  years  to  complete.  The 
program  expenditure  requirements  needed  each  year  must  be  met  for  a  program  to  finish 
on  schedule.  A  defense  program’s  R&D  budget  for  any  particular  year  is  expended  or 
outlayed  over  several  years.  The  outlay  pattern  varies  little  from  year  to  year.  The 
budget  must  be  sufficient  to  meet  expenditures  considering  the  outlay  delay.  Several 
models  aid  in  forecasting  fiscal  requirements  for  a  program  in  the  R&D  phase.  These 
models  when  scaled  fit  the  expenditures  over  time  in  constant  dollars.  Each  of  these 
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models  fit  expenditures,  which  is  the  budget  obligation  authority  that  is  spent  over  J 
years. 

Converting  Budget  Profiles  to  Expenditure  Profiles 

R&D  programs  in  the  Department  of  Defense  receive  budget  authority  or  total 
obligation  authority  (TOA)  that  may  be  spent  over  several  years.  Since  this  funding 
provides  the  resources  necessary  to  carry  out  the  program  over  multiple  years,  the  OSD 
comptroller  publishes  standard  outlay  (expenditure)  patterns  for  these  programs  as  part  of 
the  annual  budget.  The  outlay  pattern  is  the  targeted  fixed  percentage  of  funds  to  be 
spent  each  year. 

To  change  from  a  budget  to  an  expenditure  profile,  outlay  rates  are  applied  to  the 
budget.  The  result  is  the  expenditure  profile  in  then-year  dollars,  which  are  the  annual 
dollar  amounts  spent  on  the  program.  To  do  this,  the  formula  we  use  is 

O,  =  BjS]  +  Bu1S2  +  Bj.2S3  +. . .  +  BujSj  (1) 

with  B,  =  0  for  i  <  0  and  where  O,  is  the  expenditures  and  B>  is  the  authorized  budget 
amount  for  /th  year  in  then-year  dollars,  s,  is  the  outlay  rate  for  yth  year  of  the  budget,  and 
Jis  the  total  number  of  outlay  years. 

Then-year  dollars  have  an  inflation  component  built  into  them.  The  next  step  is  to 
remove  this  inflation  factor  in  the  expenditure  profile  in  order  to  obtain  the  expenditure 
profile  in  base-year  dollars.  We  divide  each  annual  expenditure  by  the  appropriate 


inflation  index  factor. 


where  Ox  is  the  expenditure  for  /th  in  base-year  dollars,  c,  is  the  inflation  factor  of  the 

given  year,  and  O,  is  the  expenditures  for  /th  year  in  then-year  dollars.  This  step  obtains 
an  expenditure  profile  of  the  program  in  base-year  dollars.  We  need  base-year  dollar 
values  because  the  expenditure  theory  applies  to  the  actual  value  accomplished. 

We  present  three  expenditure  models:  Beta,  Sech-Squared,  and  Rayleigh. 


The  Beta  Model 

Expenditures  may  be  modeled  with  the  Beta  model. 

T(a  +  b)  _ 


r(a)T@) 


•a-o 


fc-i 


where  T  is  the  gamma  function,  a>0,b>0,  and  t  =  time  (Jarvis,  1999).  Jarvis  and  Pohl 
stated,  “The  beta  curve  provides  great  flexibility;  however,  a  theoretical  justification  for 
using  the  beta  curve  is  lacking”  (Jarvis,  1999:9).  Although  the  beta  curve  can  fit  the 
expenditure  pattern  associated  with  R&D  program  expenditures  after  completion  of  the 
program,  we  know  of  no  method  to  estimate  the  model  parameters  a  and  b  before 
program  completion. 

A  useful  model  can  forecast  or  project  future  needs.  A  “good”  model  offers  the 
user  a  predictive  tool  in  this  application.  Since  the  parameters  of  the  beta  model  are 
currently  estimated  only  after  completion  of  the  program,  the  beta  curve  provides  little 
utility  in  projecting  or  forecasting  program  characteristics. 


The  Sech-Squared  Model 

Scaled  expenditure  may  also  be  modeled  using  the  Sech-Squared  model.  In 
equation  form 
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where  a  and  c  are  parameters  selected  to  fit  the  expenditures. 

Parr  notes  key  characteristics  of  this  model  as  being  symmetric  about  its 
maximum  and  infinite  tails  (Parr,  1980:294).  The  model  therefore  has  no  specific 
starting  date.  According  to  Parr,  this  illustrates  the  fact  that  before  programs  officially 
starts,  exploratory  design  studies  and  research  projects  aimed  at  specific  aspects  of  the 
task  are  attempted  (Parr,  1980:294).  Unfortunately,  these  pre-start  activities  are  not  well 
documented  nor  have  accounting  systems  been  implemented  to  track  these  preliminary 
expenditures  (Parr,  1980:295).  Although  the  Sech-Squared  Model  accounts  for  these 
activities,  little  information  actually  exists  to  support  building  funding  profiles  from  these 
pre-start  activities. 

In  sum,  the  Beta  Model  provides  little  value  as  a  predictive  tool.  The  Sech- 
Squared  Model  does  provide  predictive  capability.  The  difficulty  of  the  Sech-Squared 
model  comes  with  determining  the  parameters.  For  these  reasons,  our  attention  turns  to 
the  Rayleigh  Model. 


The  Rayleigh  Model 

Norden  noted  that  manpower  utilization  on  software  projects  mirrored  a  life-cycle 
pattern  that  follows  a  distribution  formulated  by  Lord  Rayleigh  (Putnam,  1978:347). 
Norden  proposed  using  the  Rayleigh  Cumulative  Distribution  Function  to  model  the 
manpower  utilization  during  a  project  (Norden,  1970:122).  Norden  used  the  model  to 
describe  the  quantitative  behavior  of  various  cycles  ofR&D  projects  (Norden, 
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1970:116,130).  Because  ofNorden’s  efforts,  the  Rayleigh  model  is  sometimes  referred 
to  as  the  Norden/Rayleigh  model  (Putnam,  1978:347). 

Putnam  applied  the  Rayleigh  model  to  software  system  manpower  (Putnam, 
1978:346).  He  tested  the  Rayleigh  model  against  the  man-year  budgetary  data  for  about 
50  systems  of  the  Computer  Systems  Command  and  discovered  that  the  project  follows 
this  life-cycle  model  remarkably  well  (Putnam,  1978:348). 

Putnam  was  able  to  generate  manpower,  instantaneous  costs,  and  cumulative  costs 
of  a  software  project  at  any  time  t  by  using  the  Rayleigh  equation  (Putnam,  1978:352). 
The  ability  to  generate  cumulative  costs  for  a  project  is  a  key  component  of  what  this 
thesis  effort  hopes  to  achieve.  The  following  paragraphs  provide  detail  about  how 
Putnam’s  achievement  was  adapted  to  defense  programs. 

Watkins  applied  the  Rayleigh  model  to  defense  acquisition  data  and  concluded 
the  Rayleigh  modeled  the  earned  value  of  contracts  well.  Watkins  applied  the  Rayleigh 
model  to  a  Helicopter  Engine  contract  and  a  Cruise  Missile  contract  and  found  that 
differences  in  actual  and  modeled  expenditures  were  not  statistically  significant 
(Watkins,  1982:79). 

Lee,  Hogue,  and  Gallagher  (Lee,  1997:31)  report  Lee,  Hogue,  and  Hoffman’s 
unpublished  conclusions  that  the  Rayleigh  function  modeled  outlays  of  a  wide  variety  of 
defense  acquisition  contracts.  Their  research  demonstrated  that  the  Rayleigh  Model  fits 
contract  expenditures  expressed  in  base-year  dollars  of  20  defense  R&D  acquisition 
programs.  This  substantiates  the  claim  that  cumulative  constant  dollar  R&D  expenditures 
for  defense  programs  may  be  modeled  with  the  Rayleigh  cumulative  expenditure  function 
(Gallagher,  1996:52). 
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Based  on  previous  research  (Putnam,  Watkins,  Lee,  and  Gallagher),  we  accept  the 
Rayleigh  model  is  useful  as  a  predictive  model  for  R&D  expenditures  for  defense 
programs.  The  results  of  these  studies  reasonably  affirm  justification  to  use  the  Rayleigh 
model. 

Before  turning  our  attention  to  the  Rayleigh  equations,  we  summarize  the 
Rayleigh  research.  Putnam  was  able  to  forecast  costs  for  a  software  projects  using  the 
Rayleigh  equation.  Later,  Watkins  in  one  paper  and  Lee,  Hogue,  and  Gallagher  in  a 
subsequent  paper  demonstrated  that  the  Rayleigh  function  also  models  defense  program 
expenditures.  With  this  understanding  the  theoretical  basis  for  this  thesis  effort — using 
the  Rayleigh  function  to  model  funding  curtailment — is  now  plausible.  Our  attention 
now  turns  to  the  function  of  the  Rayleigh  Model  itself. 

The  Weibull  and  Rayleigh  Functions  Are  Related.  The  Rayleigh  model  is  a 
degenerative  of  the  Weibull  model.  The  Rayleigh  Cumulative  Distribution  Function  is 

F(t)  =  l-e-at\  (3) 

where  a  is  the  scale  parameter  and  b  is  the  constant  of  2.  If  the  b  parameter  is  not  a 
constant  of  2,  the  equation  is  known  as  the  Weibull  Cumulative  Distribution  Function 
(CDF).  We  present  and  apply  the  Weibull  distribution  with  the  addition  of  a  location 
parameter  later. 

The  Cumulative  Distribution  Function.  Figure  2-1  presents  the  Rayleigh 
cumulative  distribution  function  using  (3)  with  a= 0.05  and  b= 2.  This  figure  depicts  how 
cumulative  percent  of  expenditures  for  a  specific  R&D  project  are  incurred. 
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The  Parameters  of  the  Ravleigh  Model.  The  cumulative  distribution  function 
graphed  in  Figure  2-1  uses  the  shaping  parameter  a  to  determine  the  curve.  We  use  a 
second  parameter  to  scale  the  Rayleigh  PDF  to  program  cost.  Norden  describes  the  a 
parameter  as  the  coefficient  that  determines  the  month  (time  period)  in  which 
manpower  utilization  is  greatest  (Norden,  1970: 126).  The  a  parameter  determines  the 
steepness  of  the  cumulative  distribution  function.  Figure  2-3  demonstrates  the 
increasing  steepness  when  the  a  parameter  is  increased  using  (3). 


Figure  2-3.  Effects  of  the  a  parameter  on  the  Cumulative  Distribution 

The  other  parameter,  d,  is  the  cost  scaling  parameter.  The  results  of  (3)  fall 
somewhere  between  0  and  1.  The  d  parameter  scales  this  answer  to  the  projected  cost  of 
the  program.  The  complete  Rayleigh  cost  model  is 

£(o=4-(ra(2))  (4) 

where  d  is  the  cost  of  the  program  and  the  variable  t  represents  the  time  from  inception  of 
the  R&D  effort  to  completion  (Lee,  1997:3 1). 
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The  Rayleigh  expenditure  rate  for  a  modeled  program  is  the  derivative  of  the 
Rayleigh  cost  model  (4), 

f(t)  =  2adte-a,\  (5) 

Setting  (5)  equal  to  0  and  solving  for  t  provides  the  apex  point  of  the  expenditure  rate 
with  respect  to  time.  As  the  a  parameter  increases,  the  peak  of  the  expenditure  rate  is 
earlier  and  higher  in  the  program’s  timeline — as  shown  in  Figure  2-4.  For  programs  of 
the  same  value  (in  base-year  dollars),  if  the  rate  of  spending  for  a  program  is  higher,  the 
time  for  the  program  to  reach  maturity  decreases.  The  shorter  duration  results  because 
larger  amounts  of  money  are  expended  in  the  earlier  phases  of  the  R&D  program. 


Figure  2-4.  Effects  of  the  a  parameter  on  the  Probability  Distribution 


The  expenditure  distribution  provides  key  information  about  the  modeled 
program.  Based  upon  the  shape  of  the  expenditure  distribution,  we  may  determine  the 
time  of  peak  expenditure,  the  magnitude  of  the  peak  expenditure,  and  the  projected 
program  conclusion.  By  manipulating  any  one  of  these  factors,  holding  d  constant,  one 
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can  determine  the  other  factors.  For  example,  if  the  peak  expenditure  is  changed  from 
year  3  to  year  4,  one  can  rework  the  expenditure  distribution  and  determine  a  new 
projected  project  conclusion  time  and  remaining  expenditures  assuming  the  overall 
program  cost  and  requirements  remain  constant.  A  simple  example  will  suffice. 


Figure  2-5.  Hypothetical  Modeled  Program 


Figure  2-5  presents  a  Rayleigh  cost  model  for  a  hypothetical  modeled  program 
(The  parameter  d  is  set  to  one,  so  percent  of  expenditures  are  depicted).  By  changing  the 
projected  conclusion  of  the  hypothetical  R&D  effort  from  Time  10  (Point  D)  to  Time  15 
(Point  E)  several  consequences  occur.  First,  the  peak  expenditure  period  changes  from 
approximate  Time  2.75  (Point  A)  to  approximate  Time  4.25  (Point  B).  Second,  the 
amount  at  the  peak  expenditure  time  period  is  reduced.  Finally,  the  expenditures  at  Time 
7  increase  from  previous  amounts  (Distance  C).  The  expenditures  for  all  intervals  along 
the  time  horizon  need  to  be  recalculated. 

Determining  the  a  Parameter.  The  shift  in  the  Rayleigh  probability  density 
function  signifies  that  the  a  parameter  has  changed.  Two  techniques  exist  for 
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determining  the  appropriate  a  parameter:  one  based  on  the  time  of  peak  expenditure  rate 
method  and  another  based  on  the  expected  completion  time  method.  Following  Lee, 
Hogue,  and  Gallagher,  we  describe  the  characteristics  of  the  Rayleigh  model,  explaining 
in  detail  the  implications  of  changes  in  the  cost  model  (Lee,  1997). 

The  Time  of  Peak  Expenditure  Method.  This  method  determines  the  a 
parameter  by  establishing  the  time  point  at  which  peak  expenditures  of  an  R&D  program 
occurs.  Asserting  the  point  of  peak  of  expenditure  can  be  estimated  with  some  degree  of 
reliability.  For  example,  the  peak  expenditure  rate  in  aircraft  R&D  programs  comes 
approximately  at  the  time  of  the  first  test  flight  (Lee,  1997:32).  The  time  of  peak  rate 
expenditure,  tpeak,  may  be  related  to  the  a  parameter  by  setting  the  derivative  of  Equation 
(5)  to  zero.  If  the  time  of  peak  rate  of  expenditures  is  known,  the  appropriate  shape 
parameter  a  is  determined  with  Equation  (5)  (Lee,  1997:32). 

a  =  0.5  (6) 

The  Expected  Completion  Time  Method.  The  right  side  of  the  Rayleigh 
has  an  infinite  tail.  Therefore,  a  point  must  be  determined  that  can  be  considered  the  end 
of  the  project.  Lee,  Hogue,  and  Gallagher  define  the  time  of  final  development,  tfinai,  of  a 
project  in  the  Engineering,  Manufacturing,  and  Development  (EMD)  phase  when  the 
Rayleigh  cumulative  distribution  function  shows  97%  of  the  cumulative  Research  and 
Development  (R&D)  expenditures  (expressed  in  base-year  dollars)  has  been  reached 
(Lee,  1997:32;  Gallagher,  1996:52).  In  equation  form 

D  =  E(t final)  -  0.91  d ,  (7) 
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where  D  is  the  total  R&D  program  cost  in  constant  dollars  and  d  scales  the  Rayleigh 
cumulative  distribution  function  to  costs.  Lee,  Hogue,  and  Gallagher  (Lee,  1997:33; 
Gallagher,  1996:52)  demonstrate  that  the  a  parameter  can  be  obtained  using  the  formula 

<**3.5  tfnal.  (8) 

Therefore,  given  a  projected  completion  time,  the  Rayleigh  shape  parameter  may  be 
determined  with  (8). 

In  summary,  utilizing  either  the  time  of  peak  expenditure  method  or  the  expected 
completion  time  method  determines  an  appropriate  a  parameter.  The  a  parameter  coupled 
with  the  total  program  cost  (expressed  in  base-year  dollars)  provides  enough  information 
to  use  the  Rayleigh  cost  models.  Once  the  model  parameters  are  estimated,  cost  analysts 
perform  manipulations  to  forecast  consequence  of  various  “what-if  ’  scenarios.  Our 
discussion  now  turns  to  the  characteristics  of  the  Rayleigh  Expenditure  Distribution. 

Characteristics  of  the  Rayleigh  Expenditure  Distribution.  The  Rayleigh  model’s 
initial  increase,  peak,  and  trailing  decrease  are  indicative  of  certain  characteristics  of  the 
modeled  program.  At  the  beginning  of  a  project,  defined  to  be  time  0,  no  effort  or 
expenditures  have  been  expended.  Appropriately  E(0)  =  0  ,  as  defined  in  (4). 

The  period  of  increase  in  the  Rayleigh  expenditure  rate  distribution,  the  point  beginning 
with  inception  and  ending  at  the  apex  of  the  peak,  is  determined  by  how  skill  acquisition 
is  accomplished.  The  Rayleigh  model  is  developed  as  the  product  of  two  functions.  The 
Weibull,  and  hence  Rayleigh,  cost  models  can  be  derived  from  the  assumption  that  the 
rate  at  which  work  is  completed  is  a  function  of  performance  and  remaining  work. 

Define  the  percent  work  remaining  as  w(t)  and  performance  as  p(t),  both  at  time  t.  Then 
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^P-=pw-m, 

at 


which  may  be  solved  for  w(t).  Let  z(t)  =  1  -  w(t) ,  so  —zj~  -  an<^ 

-  - p(t)z(t ) .  Integrating,  we  obtain  ln(z(7))  =  -  f  p(r)dr .  We  evaluate  both  sides 

dt  Tt0 

T  —  t 

~  !p(,T)dT 

to  the  power  of  the  base  of  the  natural  logarithm,  Z  (t )  =  6  T~°  .  We  substitute 

back  in  percent  of  work  to  obtain 

-  \p(j)dr 

M(t)-  1-e** 

We  define  performance  on  the  program  for  any  given  time  as  a  constant  multiplied  by 


T=t 


T=t 


time  to  constant  power,  p{t)  =  ctb .  Since 


f  p(r)dT=  f  cf  dr= - tM 

^  J  b+ 1  ’ 


T= 0 


F=0 


"Mf)  — 1— £  1  .  With  linear  growth  in  performance  over  time,  b=  1  and  a=c/2,  we 

obtain  the  Rayleigh  cumulative  distribution  function  shown  in  (3).  If  performance 
improves  with  time  to  a  power  other  than  one,  we  have  derived  percent  work  complete 
according  to  a  Weibull  cumulative  distribution  function. 

The  Rayleigh  model  begins  at  zero  and  has  an  infinite  right  tail.  At  the  initial 
stage,  skill  levels  are  zero.  As  time  progresses,  the  amount  of  skills,  represented  as  2at, 
available  increase  linearly.  The  a  parameter  governs  the  slope  of  the  line  as  time 
increases.  Parr  states  that  the  initially  rising  work  rate  is  due  to  linear  skills  acquisition 
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curve  governing  the  “skill”  available  for  solving  problems  (Parr,  1980:291).  The  level  of 
skills  available  to  the  project  grows  as  time  on  a  particular  project  increases. 

One  of  the  assumptions  of  the  Rayleigh  model  is  there  is  a  linear  skill  acquisition. 
Lee,  Hogue,  and  Gallagher  support  this  assumption  (Lee,  1997:32).  Resources,  money  in 
this  case,  are  needed  to  fuel  the  skills  acquisition  process. 

Similarly,  as  time  progresses,  the  percent  of  work  remaining  decreases.  As  time 
progresses  and  skills  to  solve  those  problems  increase,  fewer  and  fewer  problems  remain. 
Parr  sums  this  point  up  stating  that  the  decay  in  the  work  rate  at  the  conclusion  of  the 
project  is  due  to  the  exhaustion  of  the  problem  space  (Parr,  1980:291).  Initially,  the  skills 
acquisition  dominates  and  the  rate  of  expenditures  increases.  Eventually,  the  percent  of 
work  remaining  dominates  and  the  rate  of  expenditures  steadily  decreases.  The  peak  of 
the  Rayleigh  expenditure  rate  is  where  the  dominance  switches  from  skills  acquisition  to 
percent  work  remaining. 

Unlike  the  left  tail,  the  right  tail  continues  infinitely.  In  a  strict  sense,  this  means 
that  the  R&D  project  never  terminates.  A  never-ending  project  is  simply  not  reality.  For 
this  reason  a  completion  time  has  to  be  fixed.  Lee,  Hogue,  and  Gallagher  (Lee,  1997:32) 
fix  this  point  when  97%  of  the  expenditures  for  an  R&D  project  have  been  reached  (7). 

Figure  2-6  presents  a  graph  of  the  terms  2at  and  e~atl  separately  and  their 

product. 
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Note  the  line  2at  continually  increases  as  time  progress.  As  stated  before,  the 

graph  of  e~a‘2  decreases  monotonically.  The  probability  density  function  is  the  Product 
line  in  Figure  2-6. 

Up  to  this  point,  the  discussion  has  explored  the  Rayleigh  function,  the  function 
traits,  and  the  methods  of  determining  its  parameters.  The  Rayleigh  Model  uses  these 
characteristics  to  aid  in  forecasting  expenditures  requirements  for  programs. 

The  Rayleigh  function  has  the  shape  parameter  set  to  a  constant  of  2.  This  makes 
the  model  somewhat  rigid  in  its  ability  to  model  programs.  The  Rayleigh  function  forces 
a  proportionate  tail  using  the  peak  expenditure  point  as  the  start.  In  actuality  there  are 
programs  where  a  proportionate  tail  is  not  derived  from  the  point  of  peak  expenditures. 
For  example,  a  program  may  have  a  peak  expenditure  during  one  time  period  and  a  very 
short  tail — program  expenditures  stop  shortly  thereafter.  The  Rayleigh  function  would 
not  provide  an  accurate  model  of  reality  in  this  case  because  of  its  rigidity  tied  to  the 
constant  shape  parameter.  The  resolution  to  this  predicament  is  to  allow  the  shape 
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parameter  to  be  variable,  which  takes  us  from  the  Rayleigh  function  to  the  Weibull 
function. 


The  Weibull  Function 

The  form  of  the  Rayleigh  Equation  shown  in  (3)  is  prevalent  in  the  literature  cited 
up  to  this  point  (Norden,  1970:122;  Putnam,  1978:349;  Lee,  1997:30;  Gallagher, 
1996:52).  This  equation  assumes  that  program  funding  and  manpower  start  with  the 
inception  of  the  program.  Some  programs  begin  with  one  or  more  years  of,  in  essence, 
insignificant  funding.  Therefore,  we  model  resulting  expenditures  with  a  location 
parameter,  which  disregards  the  starting  time  with  insignificant  funding. 

In  the  Rayleigh  functions,  the  shape  parameter  fixes  the  time  of  peak  expenditures 
and  program  completion,  as  seen  in  (6)  and  (8).  Since  some  programs,  particularly  those 
being  restructured,  do  not  follow  this  fixed  pattern,  we  use  a  three-parameter  Weibull 
distribution  (Hines,  1980:165). 


F(t)  -\-e 


(9) 


where 

t  =  time 
y  =  location 
5  =  scale 
(3  =  shape 

With  the  exception  of  the  location  parameter,  (3)  and  (9)  are  equal.  The  shape 
parameter  in  (3),  b,  is  equal  to  the  shape  parameter  of  (9),  (3.  The  scale  parameter  of  (3), 
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a,  is  equal  to  1/52  of  (9).  The  time  parameter,  t,  of  (3)  is  shown  as  t  in  (9).  Equation  (9) 
introduces  the  location  parameter,  y,  to  the  Weibull  function.  The  location  parameter 
allows  the  Weibull  function  to  model  program  expenditures  when  programs  are  initiated 
but  do  not  receive  significant  funding  until  a  later  year.  The  Weibull  shape  parameter 
allows  skills  acquisitions  at  other  than  linear  rates.  In  addition,  the  Weibull  time  of  peak 
expenditures  does  not  fix  the  completion  time  of  the  program.  The  Weibull  cost  model  is 


f 


E(t)  =  d 


1-e 


Ml 


(10) 


Using  the  Weibull  function  allows  expenditure  profiles  to  be  modeled  with  a  greater 
degree  of  accuracy  than  the  Rayleigh  function. 


Relating  Expenditures  to  Funding  (Lee.  1997:34-38) 

R&D  programs  in  the  Department  of  Defense  receive  budget  authority  or  total 
obligation  authority  (TOA)  that  may  be  spent  over  several  years.  Since  this  funding 
provides  the  resources  necessary  to  carry  out  the  program  over  multiple  years,  the  OSD 
comptroller  publishes  standard  outlay  (expenditure)  patterns  for  these  programs  as  part  of 
the  annual  budget.  The  outlay  pattern  is  the  targeted  fixed  percentage  of  funds  to  be 
spent  each  year. 

Modeled  annual  outlays  (expenditures)  are  calculated  in  base-year  dollars,  defined 
as  Ot  for  the  /th  year  of  the  program.  The  hat  indicates  modeled  values  as  opposed  to 
those  derived  from  the  budget  with  (1)  and  (2).  The  difference  between  the  current  and 
previous  modeled  cumulative  expenditures  can  be  put  in  formula  form  as 
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6,  =£((,)-£((,.,)  (11) 

where  E  is  the  cumulative  expenditures  in  (10)  and  time  U  is  the  end  of  the  fiscal  year  i. 

The  Rayleigh  model  provides  the  appropriate  outlay  profile  needed  for  an  R&D 
program.  By  knowing  the  parameters  discussed  previously,  the  model  provides  an  outlay 
profile  sufficient  for  the  program  to  progress  at  a  desirable  rate.  Lee,  Hogue,  and 
Gallagher  provided  an  excellent  example  in  their  article  that,  at  this  point,  needs  to  be 
understood.  Their  example  with  some  slight  modifications  to  aid  understanding  is  as 
follows  (Lee,  1997:34): 

As  an  example,  consider  a  hypothetical  program  with  EMD  expenditures 
of  a  billion  dollars  (expressed  in  terms  of  millions  of  dollars)  occurring  over  10 
years.  The  base  year  for  the  program  is  1995.  Since  tfinal  =  10.0  and  D  =  1,000, 

the  Rayleigh  model  parameters  calculated  with  [(7)]  and  [(8)]  are  <7=  1030.93  and 
a  =  0.035.  Table  2-1  shows  the  cumulative  expenditures  calculated  with  the 
Rayleigh  model  in  [(4)]  and  these  parameters.  The  annual  outlays  were  calculated 
with  [(9)].  We  applied  Navy  raw  inflation  indices,  shown  in  the  last  column,  to 
covert  to  outlays  in  then-year  dollars. 
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Table  2-1.  Example  Outlay  Profile 


Year 

Time 

Cumulative 

Rayleigh 

(Base- Year 
Dollars) 

Annual 

Outlays 

(Base- Year 
Dollars) 

Annual 

Outlays 

(Then- Year 
Dollars) 

Inflation 

Index 

1995 

1 

35.46 

35.46 

35.46 

1.0000 

1996 

2 

134.68 

99.22 

102.20 

1.0300 

1997 

3 

278.57 

143.89 

152.65 

1.0609 

1998 

4 

442.05 

163.48 

178.64 

1.0927 

1999 

5 

601.17 

159.12 

179.09 

1.1255 

2000 

6 

738.50 

137.33 

159.20 

1.1593 

2001 

7 

845.40 

106.90 

127.65 

1.1941 

2002 

8 

921.18 

75.78 

93.20 

1.2299 

2003 

9 

970.39 

49.22 

62.35 

1.2668 

2004 

10 

999.80 

29.40 

38.37 

1.3048 

Total 

999.80 

1,128.80 

From  their  example  several  important  points  are  made.  The  Cumulative  Rayleigh 
provides  the  funding  needed  for  the  program’s  success  in  base-year  dollars 

The  annual  outlays  are  presented  in  base-year  and  then-year  dollars  in  the  fourth 
and  fifth  columns,  respectively.  In  other  words,  the  sample  program  needs  $143. 89M  in 
FY95  dollars  or  equivalently  $152.65M  in  then-year  dollars  during  the  third  year — 
program  funding  is  given  in  then-year  dollars.  One  would  expect  the  program  manager 
to  obligate  an  amount  equal  to  annual  outlay  in  then-year  dollars.  This  is  the  money  the 
program  manager  would  expend  each  year  since  these  figures  have  been  adjusted  for 
inflation. 

Up  to  this  point,  we  have  developed  an  expenditure  profile.  We  know  what  the 
program  spends  each  year.  The  task  is  how  to  convert  the  projected  expenditures  into  a 
budget  profile.  What  distinguishes  an  outlay  (expenditure)  profile  from  a  budget  profile? 
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Outlays  are  the  monies  flowing  from  a  program.  These  monies  come  from  different 
obligation  authorities  in  different  years.  The  outlay  (expenditure)  profile  is  based  on  an 
amalgamation  of  all  these  varied  years  of  funding. 

Our  approach  must  aid  in  build  the  necessary  total  obligation  authority  (TOA)  or 
budget  profile.  This  profile  is  not  a  straight  conversion  of  the  outlay  profile  developed 
above.  The  entire  appropriation  for  large-dollar  projects  cannot  be  spent  during  the  fiscal 
year  in  which  it  was  authorized.  Certain  amounts  of  the  appropriation  are  allotted  to  the 
program  each  fiscal  year. 

For  example,  the  OSD  comptroller  established  outlay  rates  of  43.10,  44.55,  8.75, 
and  3.60  (expressed  as  percents)  for  the  first  through  fourth  year  respectively  in  the  FY 
2000  National  Defense  Budget  for  RDT&E  programs  (Outlay  Rates,  FY2000).  Although 
a  program  may  have  $10M  appropriated  to  it  in  a  year,  the  program  manager  is 
encouraged  only  to  obligate  (expend)  $4.3M  during  the  first  year  because  the  OSD 
comptroller  established  a  43.10%  outlay  for  the  first  year.  During  the  second  year  of  the 
program,  the  program  manager  obligates  about  $4.45M  from  the  first  year  budget 
authority.  If  the  program  manager  needs  $10M  in  total  expenditures  for  the  second  year 
of  the  program,  he  should  request  $12.88M  in  appropriations  for  the  second  year.  The 
calculation  is  as  follows 

($  10M  needed  -  $4.45M  from  first  year  appropriations)  _  „  001,  , 

- — -  —  JplzLooJVl. 

0.43 10  first  year  outlay  rate 

The  type  and  complexity  of  the  calculations  described  above  increase  in 
proportion  to  the  length  and  size  of  the  R&D  program.  From  the  simple  example  above, 
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it  can  be  noted  that  balancing  the  OSD  comptroller  outlay  rates  and  the  different  years’ 
authorizations  become  more  difficult  as  the  program  continues  on  in  years. 

It  should  also  be  apparent  the  complex  task  inherent  in  developing  a  budget 
profile.  The  outlay  pattern  can  be  easily  forecasted  with  a  Rayleigh  or  Weibull  model. 
The  challenge  comes  in  translating  the  outlay  pattern  into  a  budget  profile  taking  into 
account  various  appropriation  years  tied  to  different  outlay  rates.  The  analyst  knows 
what  outcome  is  wanted  (the  expenditure  profile)  and  must  now  develop  a  budget  to 
match  the  expenditure  profile.  Lee,  Hogue,  and  Gallagher  describe  an  approach.  We 
present  this  as  an  integral  part  of  our  approach. 

Their  method  uses  constrained  nonlinear  estimation  to  build  the  budget  profile.  In 
laymen’s  terms,  this  means  that  we  change  various  inputs  simultaneously  that  may 
impact  other  inputs  until  an  optimal  solution  is  reached.  They  substitute  budget  estimate 

Bt  into  (1)  and  (2).  We  select  the  budget  estimate  that  minimizes  the  sum  of  squares 

error  between  the  projected  outlay  profile  based  on  the  budget  and  the  projected  Weibull 
expenditure  values.  We  select  the  revised  budget  value  that 

(12) 

j=i 

A  A 

subject  to  Bt  1 0  and  where  Ot  is  the  outlay  (expenditure)  profile  needed  as  computed 

using  the  Weibull  cost  model  in  (9)  and  (10)  and  Oi  is  the  budget  outlay  for  each  z'th  year 
from  (1)  and  (2).  N+J-l  is  the  total  program  and  outlay  years  we  are  trying  to  calculate. 
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Chapter  Summary 

In  summary,  the  Rayleigh  model  aids  in  determining  an  expenditure  profile  for  an 
R&D  program.  This  assumption  is  based  on  several  fundamental  insights  detailed  in  this 
chapter.  First,  Putnam  derived  cumulative  costs  of  a  software  project  using  the  Rayleigh 
function.  Lee,  Hogue,  and  Gallagher  concluded  the  Rayleigh  function  modeled  outlays 
of  a  wide  variety  of  defense  acquisition  programs.  Therefore,  using  the  Rayleigh  and  its 
generalization,  the  Weibull  function,  is  justified  in  modeling  expenditure  profiles. 

This  chapter  also  discusses  the  Weibull  function  and  the  flexibility  it  provides  in 
modeling  funding  profiles.  We  explain  how  different  parameters  of  the  Weibull  function 
are  derived.  We  follow  previous  research  efforts  to  define  the  end  of  a  program  as  when 
97%  of  modeled  funds  are  expended. 

We  discuss  expenditure  profiles  and  budget  profiles  and  the  importance  that 
outlay  rates  play  evolving  an  expenditure  profile  from  a  budget  profile.  When  program 
funding  levels  change,  a  new  Rayleigh  or  Weibull  model  can  be  developed  along  with  the 
associated  funding  profile. 

In  the  next  chapter  we  model  the  expenditure  profile  of  a  program  before  funding 
curtailment.  The  parameters  derived  in  modeling  the  expenditure  profile  serve  as  the 
starting  parameters  for  developing  the  parameters  for  the  expenditure  profile  of  the 
budget  with  the  funding  curtailment.  The  approach  of  Lee,  Hogue,  and  Gallagher  is  used 
to  determine  a  corresponding  budget.  Once  this  is  accomplished,  the  original  budget  up 
to  the  time  period  of  funding  curtailment  is  combined  with  the  budget  derived  from  the 
model.  This  combined  budget  is  in  then-year  dollars  and  provides  the  user  with  the  cost 
consequences  of  the  funding  curtailment. 
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III.  Research  Methodology 


Chapter  Overview 

Studies  suggest  the  Rayleigh  Model  has  predictive  capability  in  large  dollar 
procurements  that  are  in  the  research  and  design  (R&D)  phase.  This  research  develops  a 
new  R&D  budget  when  funds  for  an  on-going  program  are  curtailed.  Describing  the 
methodology  for  this  research  is  the  focus  of  this  chapter. 

Program  expenditures  have  been  shown  to  follow  the  Rayleigh  probability  density 
function.  When  the  program’s  funding  profile  changes,  the  associated  Rayleigh 
probability  density  function  changes.  The  task  is  to  tie  the  original  Rayleigh  probability 
density  function  to  the  proposed  Rayleigh  density  function  taking  into  account  that  time 
and  expenditure  has  already  been  incurred. 

This  chapter  begins  with  an  overview  of  the  staggered  funding  profiles  when 
funding  curtailment  is  proposed.  Afterward,  this  chapter  describes  the  proposed 
methodology  that  provides  the  budget  needed  with  considering  funding  curtailment.  The 
pieces  of  information  needed  and  the  steps  to  apply  the  methodology  are  outlined.  This 
chapter  concludes  with  a  brief  summary  of  the  methodology. 

Curtailment  Implications  and  Consequences 

When  funding  cuts  are  directed  or  being  considered,  a  new  budget  profile  is 
calculated  with  different  appropriations  amounts  being  calculated  with  the  outlay  rates. 
The  program  manager  may  model  the  expenditures;  the  problem  is  determining  the 
necessary  funding  by  year  with  the  current  year’s  appropriations  being  reduced.  The  task 
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is  to  create  a  model  that  allows  these  “budget  drills”  to  be  accomplished  quickly  and  with 
relatively  good  accuracy. 

A  simple  example  provides  an  illustration  of  the  complexities  such  a  possible 
curtailment  has  in  projecting  future  budgets.  Figure  2-1  shows  a  cumulative  expenditure 
profile  of  a  program.  We  derived  the  parameters  for  this  fictional  program  using  (10) 


with  d=  100,  a  - 

102 
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—  =  0.035  (7),  and  b=2.  Now  suppose  the  program  modeled  in 


Figure  2-1  is  under  consideration  for  funding  curtailment  in  time  period  6  as  shown  in 
Figure  3-1. 


Figure  3-1  shows  a  new  dashed  cumulative  expenditure  line  with  a  new  final  time 
for  the  project  due  to  the  proposed  funding  curtailment.  Had  the  curtailment  been  know 
from  the  project  onset,  the  project  would  have  had  an  expenditure  profile  of  the  dashed 
line  instead  of  the  solid  line.  The  problem  becomes  complex  because  the  program  has 
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actually  spent  money  at  the  rate  of  the  solid  line  (Original  Expenditure)  up  to  time  period 
6  and  should  now  spend  money  at  the  rate  indicated  by  the  dashed  line  for  all  remaining 
time  periods. 


Figure  3-2.  Staggered  Efficiency  Profile 
Figure  3-2  presents  the  staggered  efficiency  profile.  The  program  progresses 
along  the  solid,  thick  line  until  year  six.  At  year  six  the  curtailment  occurs.  The  shaded 
triangle  represents  effort,  which  cannot  be  utilized  by  the  program.  There  are  insufficient 
funds  available  to  take  advantage  of  the  work  previously  done  and  some  earned  value  is 
lost.  An  example  of  such  work  is  an  engineering  task  that  cannot  be  continued.  The 
program  does  not  receive  the  benefit  of  the  task  because  the  funding  is  curtailed.  When 
the  task  is  taken  up  again,  effort  is  needed  to  reacquaint  the  engineers  with  the  work 
previously  done.  We  assume  the  task  has  to  be  accomplished  because  the  requirement 
driving  the  task  is  still  valid. 
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Figure  3-3.  Staggered  Budget  Profile 


Figure  3-3  presents  the  staggered  budget  profile.  Both  figures  show  the  difficulty 
facing  the  program  manager.  The  solid  thick  line  represents  the  staggered  budget  profile 
the  program  now  faces.  The  program  has  received  a  budget  following  the  top  curve  up 
until  time  period  6.  After  time  period  6,  the  remainder  of  the  program  follows  the 
resulting  budget  curve. 


Figure  3-4.  Expenditure  Profile  with  Budget  Curtailment 
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Figure  3-4  presents  the  cumulative  expenditure  profile  for  a  sample  program  in 
base-year  dollars  with  the  funding  curtailed  in  the  seventh  year.  This  figures  shows  the 
schedule  growth  from  approximately  year  16  to  year  19.  We  also  note  the  increase  in 
expenditures  for  this  program  from  approximately  $280M  to  $310M  base-year  dollars. 

The  purpose  of  this  thesis  effort  is  to  propose  an  approach  that  could  quickly 
estimate  the  cost  of  the  latter  half  of  the  staggered  expenditure  profile  curve  taking  into 
consideration  the  expenditures  already  made  by  the  program  and  the  amount  of  the 
proposed  curtailment. 

In  our  proposed  approach,  the  decrease  at  curtailment,  year  6  in  Figure  3-2, 
represents  lost  effort.  This  lost  results  from  efforts  interrupted  and  delayed.  In  Appendix 
A,  we  present  an  alternative  approach  that  allows  the  expenditures  to  smoothly  transition 
to  the  new  outlay  profile  without  any  lost  effort. 

Information  Needed 

The  proposed  methodology  requires  three  pieces  of  information:  the  original 
budget  profile,  the  revised  time  to  complete  the  R&D  program,  and  the  amount  and  year 
of  the  budget  cut.  The  first  piece  of  information  needed  is  the  current  budget  profile 
before  the  proposed  curtailment.  The  program  manager  or  program  cost  personnel  keeps 
this  information.  They  report  it  to  congress  in  their  annual  SAR  reports. 

The  second  piece  of  information  is  to  determine  the  revised  final  time,  t™nal .  We 

use  Lee,  Hogue,  and  Gallagher’s  definition  of  tf,„ai  given  in  (7).  Applying  the  expenditure 
outlay  rates  to  the  current  budget  results  in  a  by-year  expenditure  profile.  We  estimate 
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the  Weibull  model  parameters  for  this  distribution.  Based  on  these  parameters  and  (7), 
we  determine  tr^nal . 

In  practice  the  revised  final  time,  tjnal ,  may  be  determined  in  two  ways.  The  first 
and  most  convenient  is  that  the  user  specifies  the  final  time.  In  our  tests,  we  specify  the 
revised  final  time,  tjnal ,  as  the  t°^al  for  the  original  budget  increased  by  the  number  of 

additional  years  of  program  R&D  funding. 

The  other  method  involves  spending  caps.  A  program  reviewer  may  determine 
the  annual  amounts  of  funding  for  a  particular  program  are  too  high.  Instead  of  focusing 
on  the  completion  time  of  the  project,  the  aim  is  to  not  exceed  a  certain  expenditure 
amount  during  any  program  year.  If  this  is  the  case,  the  experimenter  selects  a  trJ?al  and 
applies  our  method.  If  the  revised  budget  exceeds  the  ceiling  amount  for  any  year,  the 
final  time,  t™na] ,  is  extended.  The  same  process  is  reiterated  until  the  revised  budget  no 
longer  exceeds  the  funding  ceiling. 

The  final  piece  of  information  needed  to  forecast  the  budget  profile  when 
considering  funding  curtailment  is  the  amount  of  the  proposed  curtailment.  This  figure  is 
usually  reported  in  budget  figures  and  not  in  expenditure  figures.  For  the  proposed 
methodology,  budget  figures  suffice. 

The  Methodology 

With  these  three  pieces  of  information  (the  original  budget,  revised  final  time,  and 
funding  curtailment),  and  analyst  may  apply  our  approach.  The  budget  for  a  program  is 
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reported  in  then-year  dollars.  The  first  step  in  the  methodology  is  to  change  the  budget 
then-year  profile  into  the  expenditure  base-year  profile  using  (1)  and  (2). 

The  next  step  is  to  fit  the  Weibull  model  by  estimating  the  five  parameters  in  (10) 
for  the  original  expenditure  profile.  We  find  the  Cost  parameter  by  taking  the  summed 
total  of  the  base-year  expenditures  and  dividing  by  0.97  based  on  (7).  The  short  reason  is 
because  when  the  Rayleigh  or  Weibull  profile  obtains  a  cumulative  value  of  97%,  we 
define  the  R&D  program  to  be  complete.  The  Location,  Shape,  and  Scale  parameters  are 
estimated  based  on  the  yearly  expenditure  data.  We  conduct  a  nonlinear  search  to 
minimize  the  squared  error  between  the  modeled  and  cumulative  data  to  derive  the 
Location,  Shape,  and  Scale  parameters.  The  original  t°^al  is  determined  based  on  the 
modeled  expenditures  with  (7)  and  (10)  as 

t  final  =<?(-ln(0.03))*  +  y.  (13) 

The  t™nal  parameter  is  needed  to  derive  the  revised  Weibull  parameters  and  is 
calculated  by  adding  the  number  of  additional  budget  years  to  the  original  t°^al  for  our 
tests. 

Each  time  period  of  the  program  correlates  to  a  certain  expenditure.  For  example, 
the  third  year  of  a  program  may  have  $12M  in  expenditures  for  that  year  alone  and  $18M 
in  cumulative  expenditures  from  the  first  three  years.  The  Weibull  function  also  provides 
an  expenditure  amount  for  every  time  period,  usually  years,  U  Subtracting  cumulative 
expenditures  of  the  year  under  consideration  from  the  previous  year  gives  the  yearly 
expenditure  amounts.  Each  time  period  of  the  program,  usually  given  in  years,  has  a 
cumulative  expenditure  amount  correlated  to  it.  By  manipulating  the  Location,  Shape, 
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and  Scale  parameter,  the  amounts  provided  by  the  Weibull  function  change  to  be  come 
closer  and  closer  to  the  actual  amounts.  The  method  we  proposed  is  to  minimize  the  sum 
of  the  square  difference  between  the  actual  expenditure  amounts  and  expenditure 
amounts  generated  by  the  Weibull  function.  We  select  the  Weibull  parameters  y ,  5,  and 
(B  to 

min  S(0(-0,)  04) 

i=l 

where  N  is  the  number  of  budget  years,  J  is  the  number  of  outlay  years,  and  0°rg  is  the 
expenditures  for  rth  year  in  base-year  dollars  from  (1)  and  (2)  and  0°rg  is  the  expenditure 

amount  generated  by  the  Weibull  function  in  base-year  dollars  from  (10)  and  (1 1). 

We  use  Microsoft  Excel’s  Solver  feature  as  the  primary  utility  for  our 
methodology.  Several  other  nonlinear  search  packages  could  also  solve  this  problem. 

The  target  cell  is  the  sum  of  all  of  the  squared  error.  Solver  is  set  to  minimize  this  value. 
Solver  manipulates  the  cells  containing  the  Location,  Shape,  and  Scale  parameters  to  find 
the  minimum  value  of  the  sum  of  squared  errors. 

The  initial  values  for  the  search  are  important.  If  the  initial  values  are  too  far  off 
from  the  global  optimal  values,  solver  may  stop  searching  at  a  local  optimum.  We  used 
zero  for  the  initial  value  for  the  Location  parameter.  This  assumes  the  program  starts  in 
its  first  year  of  funding.  Several  programs  had  a  few  years  of  little  funding  after  which 
they  did  receive  significant  funding.  We  use  2  as  the  initial  value  for  the  Shape 
parameter.  Chapter  Two  discusses  that  the  Rayleigh  model  provides  for  efficient  funding 
of  a  program,  and  the  Rayleigh  model  is  a  Weibull  function  with  a  Shape  parameter  of  2. 
By  starting  the  search  for  a  Shape  parameter  at  2,  the  assumption  is  the  program  funding 
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supports  Rayleigh  expenditures.  The  initial  value  of  the  Scale  parameter  should  be  set  so 
as  to  extend  the  duration  of  the  Weibull  model  to  the  duration  of  the  actual  program.  The 
estimation  step  produces  the  Location,  Shape,  and  Scale  parameter  for  the  original 
expenditure  profile.  The  final  time  of  the  original  program  is  calculated  with  (13). 

The  next  step  is  incorporating  the  proposed  budget  cut  and  the  revised  program 
duration.  We  substitute  the  proposed  budget  in  the  appropriate  year  into  (1).  A  new 
expenditure  profile  is  created  using  this  proposed  budget  up  to  the  year  of  curtailment — 
hereafter  referred  to  as  the  revised  expenditure  profile.  For  example,  if  the  budget  in  year 
6  of  program  is  under  consideration  for  curtailment,  we  build  a  partial  expenditure  profile 
using  the  original  budget  values  for  years  1  through  5  and  the  proposed  budget  for  year  6. 

The  partial  revised  expenditure  profile  should  be  equal  in  dollar  amounts  for  each 
year  before  the  curtailment  year.  Because  the  outlay  rate  and  inflation  affect  on  the 
budget,  we  do  not  find  a  dollar  for  dollar  reduction  difference  in  the  expenditure  profile. 

The  next  step  is  to  fit  the  Weibull  Shape  and  Scale  parameters  to  this  revised 
expenditure  profile  while  retaining  the  Location  and  Cost  parameters  at  their  original 
values. 

The  t™nal  parameter  for  the  revised  budget  is  the  original  tfmai  parameter  plus  the 
increase  in  program  duration.  The  unit  of  time  should  be  consistent.  If  the  original  tfinai 
parameter  is  given  in  years  and  the  revised  t™nal  parameter  is  given  in  months,  the  revised 

tr™nal  parameter  will  need  to  be  converted  to  years. 

We  use  a  similar  method  of  fitting  the  Weibull  Shape  and  Scale  parameters  as 
determining  the  original  Weibull  parameters.  Instead  of  minimizing  the  sum  of  all  the 
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squared  errors  as  was  previously  done,  the  objective  is  to  minimize  the  squared  error  for 
the  year  of  the  proposed  budget  curtailment.  Also  the  actual  budget  expenditures  must 
exceed  the  Weibull  forecasts  for  any  year  prior  to  the  proposed  curtailment. 

We  estimate  revised  Weibull  shape  and  scale  parameters  with 

min  {dr -dr}  (15) 

subject  to 

dr  <  0°;z  fory  =  1,  ...,/ 

( final  ~  t final  (13) 

We  use  the  original  location,  y ,  and  cost  scale,  d,  parameters 

The  initial  values  for  the  Shape  and  Scale  parameters  are  the  Shape  and  Scale 
parameters  from  the  original  budget.  Since  the  original  and  revised  profiles  support  the 
same  program,  we  expect  the  parameter  values  are  close  to  the  original  parameters.  This 
estimation  obtains  the  Shape  and  Scale  parameters  of  the  revised  expenditure  profile. 

The  next  step  is  to  use  the  revised  parameters  along  with  the  Location,  Cost,  and 

t/Zai  parameters  and  build  an  expenditure  profile.  The  Weibull  function  provides  a 

cumulative  expenditure  profile.  By  subtracting  previous  time  period  cumulatives  from 
each  time  period,  the  expenditures  for  just  one  time  period  can  be  obtained  with  (11). 
The  resulting  modeled  expenditure  profile  is  in  base-year  dollars. 

The  next  step  is  to  build  a  budget  from  this  profile.  In  Chapter  Two  the 
complexities  of  taking  an  expenditure  (outlay)  profile  and  creating  a  budget  profile  are 
discussed. 
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Up  to  the  year  when  funding  curtailment  is  possible,  the  budgeted  amount  for 
each  year  has  already  been  authorized  and  possibility  spent.  The  objective  is  to  build  a 
budget  taking  into  account  what  has  already  actually  been  appropriated  with  what  effect 
of  the  curtailment.  At  this  point  we  know  what  the  budget  was  up  to  the  year  under 
consideration,  we  know  what  the  proposed  budget  will  be  for  the  year  in  consideration, 
what  we  need  is  the  budget  for  the  year  following  the  curtailment. 

We  solve  for  the  budget  in  the  out  years  that  follows  the  revised  Weibull  cost 
model  with  (12).  The  revised  budget  profile  incorporates  the  original  budget  from  up  to 
the  year  considered  for  curtailment,  the  curtailed  budget,  and  the  revised  budget  estimates 
for  the  remaining  years.  We  create  a  revised  budget  as 

{ (IS) 

The  result  of  this  step  is  a  budget  that  incorporates  the  money  already  spent,  the 
curtailment  of  funds,  and  the  remaining  budget  necessary  after  the  proposed  curtailment. 
In  summary,  the  methodology  is: 

•  Change  budget  profile  into  an  expenditure  profile  using  outlay  rates  with 

(1) 

•  Change  expenditure  profile  into  base-year  dollar  by  dividing  each  year  by 
the  appropriate  inflation  factor  with  (2) 

•  Derive  Weibull  parameters  from  the  original  expenditure  profile 

o  Cost  parameter  d  as  total  program  cost  in  base-year  dollars  divided 
by  0.97  based  on  (7) 

o  Least  squares  estimation  for  Shape,  Scale,  and  Location  parameters 
with  (14) 

o  Final  time,  t°j£al ,  based  on  the  estimated  parameters  with  (13) 
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•  Incorporate  the  proposed  budget  curtailment  and  determine  new  outlay  in 
current  year  with  (1) 

•  Specify  revised  completion  time,  we  add  the  number  of  additional  budget 
years  to  the  original  tr^nal 

•  Estimate  Weibull  Shape  and  Scale  parameters  subject  to  revised 
expenditure  less  than  original  expenditure  and  meeting  revised  completion 
time  with  (15) 

•  Develop  a  new  budget  for  revised  expenditures  with  (12) 

•  Build  a  revised  budget  using  money  already  spent  for  past  years,  proposed 
curtailment  for  the  planned  year,  and  the  new  budget  for  future  years  as 
shown  in  (16) 


Chapter  Summary 

This  chapter  describes  the  proposed  methodology  that  determines  a  revised 
budget  for  curtailment  of  an  ongoing  R&D  program.  This  chapter  contains  an 
explanation  of  each  step  along  with  a  short  summary  of  the  methodology. 

In  Chapter  Two  we  discuss  the  theory  behind  Rayleigh  and  Weibull  function  as  it 
pertains  to  defense  acquisition.  In  the  following  chapter,  Chapter  Four,  we  provide  the 
results  of  applying  this  methodology  to  actual  programs. 
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IV.  Results  and  Discussion 


Chapter  Overview 

Chapter  Three  outlines  our  methodology  to  forecast  the  budget  needed  when 
considering  funding  curtailment.  This  chapter  explains  the  program  selection  criteria  to 
test  this  methodology  and  presents  the  different  programs  used  to  validate  the 
methodology.  The  definition  of  research  success  is  also  discussed.  For  each  program, 
we  provide  the  amount  of  the  curtailment  and  the  schedule  slip  occurring  at  the  time  of 
the  curtailment.  The  Weibull  parameters  used  in  modeling  the  programs  are  given  for 
both  the  original  budget  profile  and  the  profile  incorporating  the  curtailment  (labeled 
‘revised’).  This  chapter  concludes  with  synopsized  results  from  applying  the 
methodology  to  historical  completed  programs. 

Program  Criteria 

This  section  lists  and  provides  rationale  for  programs  that  are  candidates  to 
validate  the  proposed  methodology. 

The  first  criterion  is  the  initial  program  Selected  Acquisition  Report  (SAR)  is 
after  the  early  1980s.  The  Selected  Acquisition  Report  is  submitted  to  congress  on 
certain  large  dollar  programs.  Before  the  early  1980s,  the  format  of  the  SAR  was  not 
standardized.  After  the  early  1980s,  the  budget  data  was  reported  in  a  tabular  format  that 
allows  comparison  from  year  to  year.  We  select  only  programs  that  adopted  the 
standardized  format. 
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The  second  criterion  is  the  program  completed  RDT&E  funding  before  1999. 

This  criteria  allows  us  to  know  how  the  final  RDT&E  funding  concluded  for  the 
program — which  provides  a  baseline  that  we  can  compare  all  previous  reports.  We 
considered  comparing  against  the  next  year  Selected  Acquisition  Report,  but  we  found 
the  budget  a  year  after  a  curtailment  often  constituted  an  underfunded  the  program. 

The  third  criterion  is  funding  curtailment  occurred.  Funding  must  be  delayed 
from  at  least  one  year  to  later  in  the  program.  We  found  these  programs  by  comparing 
the  annual  RDT&E  tables  with  the  final  SAR. 

A  fundamental,  critical  assumption  is  that  scope/requirements  for  the  program 
have  not  changed.  A  requirements  change  limits  the  ability  to  assess  any  funding 
increase  directly  attributable  to  delaying  money  from  one  year  to  latter  years.  We 
eliminate  programs  that  had  significant  decrease  in  base-year  dollar  totals  since  this 
indicates  a  reduction  in  program  requirements.  In  addition,  if  the  SAR  indicates  a  change 
in  requirements,  we  eliminate  the  program. 

These  criteria  provide  the  guidelines  in  selecting  programs  to  test  the 
methodology.  We  treat  the  sum  of  each  department’s  portion  of  joint  programs  as  a 
single  program.  This  prevents  examining  inter-service  transitions  of  funding  for  joint 
programs.  We  applied  the  lead  services  outlay  rates,  shown  in  Table  4-1.  Each  row  is 
the  outlay  rate  over  the  fiscal  years.  Inflation  factors  are  contained  in  Appendix  B  for 
these  programs. 


4-2 


Table  4-1.  Service  Outlay  Rates 


Air  Force 
FY2001 

FY2000 

FY1999 

FY1998 

FY1997 

FY1996 

FY1995 

FY1994 

FY1993 

Average 

StDev 

59.5 

58.8 

59.1 

50.7 

45.8 

46.3 

46.5 

46.5 

50.8 

51.56 

5.98 

33.7 

34.5 

33.1 

37.4 

39.9 

39.1 

38.8 

38.8 

34.5 

36.64 

2.67 

3.6 

3.6 

5.3 

6.8 

8.9 

8.9 

8.8 

8.9 

9.5 

7.14 

2.40 

1.0 

1.0 

1.4 

3.0 

3.6 

3.6 

3.6 

3.6 

3.4 

2.69 

1.19 

0.3 

0.3 

0.4 

0.8 

1.1 

1.1 

1.1 

1.1 

1.1 

0.81 

0.37 

0.3 

0.3 

0.3 

0.3 

0.4 

0.4 

0.4 

0.4 

0.35 

0.05 

0.2 

0.1 

0.2 

0.2 

0.18 

0.05 

98.1 

98.5 

99.8 

99.0 

99.7 

99.4 

99.4 

99.3 

99.9 

99.37  J 

Army 

FY2001 

FY2000 

FY1999 

FY1998 

FY1997 

FY1996 

FY1995 

FY1994 

FY1993 

Average 

StDev 

57.5 

56.8 

58.0 

58.0 

58.0 

57.0 

57.0 

55.0 

55.0 

56.92 

1.19 

32.5 

33.7 

33.0 

33.0 

33.0 

34.0 

34.0 

34.0 

34.0 

33.47 

0.59 

6.3 

5.0 

5.3 

5.3 

5.3 

5.3 

5.3 

7.3 

7.3 

5.82 

0.91 

2.1 

2.1 

2.1 

1.8 

1.8 

1.8 

1.8 

1.8 

1.8 

1.90 

0.15 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.8 

0.80 

0.00 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.5 

0.50 

0.00 

0.2 

0.2 

0.2 

0.2 

0.20 

0.00 

99.2 

98.9 

99.9 

99.4 

99.6 

99.4 

99.6 

99.4 

99.6 

99.61  I 

Navy 

FY2001 

FY2000 

FY1999 

FY1998 

FY1997 

FY1996 

FY1995 

FY1994 

FY1993 

Average 

StDev 

59.5 

59.3 

60.5 

58.0 

55.9 

55.9 

54.0 

55.0 

55.0 

57.01 

2.35 

31.4 

33.6 

32.5 

33.1 

31.5 

31.5 

32.4 

33.4 

33.4 

32.53 

0.89 

5.9 

4.5 

4.5 

5.4 

8.2 

8.2 

8.0 

7.8 

7.8 

6.70 

1.61 

1.9 

1.0 

1.0 

2.0 

2.0 

2.0 

2.0 

1.3 

1.3 

1.61 

0.45 

0.7 

0.3 

0.3 

0.6 

1.1 

1.1 

1.1 

1.1 

1.1 

0.82 

0.35 

0.2 

0.2 

0.2 

0.4 

0.4 

0.4 

0.4 

0.4 

0.33 

0.10 

0.6 

0.2 

0.2 

0.33 

0.23 

99.4 

98.9 

99.0 

99.3 

99.7 

99.1 

98.1 

99.0 

99.2 

99.34  | 

We  used  the  average  outlay  rate  shown  in  the  second  column  from  the  right  as  the 
outlay  rates  for  our  programs.  There  was  not  a  significant  difference  in  the  outlay  rates 
from  year  to  year.  The  first  number  in  the  average  column  is  for  the  first  year,  the  second 
number  is  for  the  second  year,  and  so  on.  The  averages  do  not  sum  to  100%  percent.  A 
small  percent  of  money  expires  without  being  obligated. 

Each  programs  inflation  index  is  contained  in  Appendix  B. 
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Success  Defined 


The  purpose  of  the  methodology  is  forecast  a  revised  R&D  budget.  The 
forecasted  budget  is  compared  to  the  final  RDT&E  budget  the  deviation  is  reported.  We 
define  methodology  success  minimal  deviation  from  the  final  reported  RDT&E  budget. 

In  formula  form 

^ Model  ^  ^ Final  )  (17) 

2--® Final 

where  B Model  is  the  then-year  dollar  revised  (Weibull)  budget  and  Bpmai  is  then-year 
budget  of  the  final  R&D  Selected  Acquisition  Report.  Method  success  is  also  calculated 
and  reported  in  the  chapter  in  base-year  dollars. 

For  example,  if  the  model  forecasts  $1 15M  as  the  budget  in  then-year  dollars  and 
the  final  RDT&E  budget  in  the  SAR  contains  $120M  in  then-year  dollars,  then  the  model 
forecasted  budget  is  -4.17%  below  (underestimated)  the  actual  final  RDT&E  budget 
SAR.  A  reporting  of  0%  would  mean  the  forecasted  amount  was  exactly  equal  to  the 
actual  final  RDT&E  budget  SAR  amount — this  would  be  a  perfect  forecast,  which  is  the 
goal. 

Programs 

We  selected  programs  based  on  the  criteria  outlined  above.  We  considered  37 
programs  that  had  their  initial  budgets  in  standard  format  from  the  1999  SAR  database. 
Fourteen  of  these  programs  report  a  complete  RDT&E  phase  before  the  1999  SAR.  Of 
these  programs,  we  found  five  programs  that  show  funding  curtailment  in  one  year  and  a 


4-4 


return  of  funding  in  later  years.  Since  one  programs  had  repeated  curtailments,  we  found 
six  data  points. 

The  MLRS-TGW  Program.  The  MLRS-TGW  program  is  an  Army  Missile 
program.  The  first  SAR  report  for  this  program  was  1984  with  the  final  SAR  for  the 
RDT&E  budget  submitted  in  the  1991  SAR. 

Table  4-2.  MLRS-TGW  Program  1985  SAR  Then-Year  Budget 
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Table  4-2  and  4-3  present  the  1985  SAR  budget  in  then-year  and  base-year 
figures  respectively.  The  base  year  budget  is  derived  using  (2)  and  the  inflation  factors  in 
Table  B-7.  It  shows  funding  curtailment  compared  with  the  final  SAR.  In  years  starting 
with  1988  through  1990  there  is  a  total  curtailment  of  $47. 4M  (then-year  dollars).  The 
1985  SAR  shows  $59.2M  is  added  back  into  the  program  in  the  years  1991  and  1992 
(then-year  dollars).  The  program  experienced  a  one-year  schedule  slip. 


Table  4-4.  MLRS-TGW  Program  Weibull  Parameters 


Program 

Original 

Revised 

Year  Location  Shape  Scale  t  final 

Year  Location  Shape  Scale  t_final 

MLRS-TGW 

1985  |  2.0273061  3.24656|  7.799445|  13.50612 

1985  |  2.027306|  3.103132  |  8.328803|  14.50612 

Table  4-4  above  presents  the  Weibull  parameters  for  the  1985  SAR  budget  before 
the  curtailment  (Original)  and  when  the  curtailment  is  considered  (Revised).  The 
location  parameter  remains  the  same  because  the  beginning  of  the  program  cannot 
change.  The  change  in  the  shape  parameter  indicates  the  peak  expenditure  for  this 
budget  profile  is  now  proportionally  earlier  than  originally  programmed  but  the  program 
is  longer.  The  scale  parameter  growth  is  indicative  of  the  increased  program  duration. 
We  increase  the  tT^nal  parameter  by  1  year  for  the  additional  budget  year. 

The  method  forecasts  1.25%  in  then-year  dollars  and  0.23%  in  base-year  dollars 
under  the  actual  final  budget  for  the  MLRS-TGW  program.  For  this  program,  the 
proposed  method  estimates  a  slightly  larger  (optimistic)  budget. 
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The  ADCAP  Program.  The  ADCAP  program  is  a  Navy  Munitions  program.  The 


first  SAR  report  for  this  program  was  1985  with  the  final  SAR  for  the  RDT&E  budget 
submitted  in  the  1994  SAR. 


Table  4-5.  ADCAP  Program  1986  SAR  Then-Year  Budget 


Budget 

Year 

1986 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1979 

17.9 

17.9 

17.9 

1980 

52.6 

52.6 

52.6 

1981 

90.6 

90.6 

90.6 

1982 

154.4 

154.4 

154.4 

1983 

180.4 

180.4 

180.4 

1984 

173.2 

172.9 

-0.3 

173.2 

1985 

125.9 

125.5 

-0.4 

125.9 

1986 

60.7 

60.3 

-0.4 

60.7 

1987 

58.0 

56.5 

-1.5 

58.0 

1988 

32.2 

20.2 

..  ..  mm 

20.2 

1989 

30.3 

26.2 

-4.1 

34.0 

1990 

7.5 

34.0 

26.5 

21.2 

1991 

29.9 

56.7 

26.8 

20.6 

1992 

42.5 

14.7 

-27.8 

17.7 

1993 

0.0 

28.0 

28.0 

11.9 

1994 

0.0 

26.7 

26.7 

19.8 

TOTAL 

1056.1 

1117.6 

61.5 

1059.0 
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Table  4-6.  ADCAP  Program  1986  SAR  Base-Year  Budget 


Budget 

Year 

1986 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1979 

26.0 

26.0 

26.0 

1980 

68.6 

68.6 

68.6 

1981 

106.7 

106.7 

106.7 

1982 

170.1 

170.1 

170.1 

1983 

189.9 

189.9 

189.9 

1984 

175.5 

175.2 

-0.3 

175.5 

1985 

123.4 

123.0 

-0.4 

123.4 

1986 

58.1 

57.7 

-0.4 

58.1 

1987 

53.1 

51.7 

-1.4 

53.1 

1988 

28.7 

18.0 

-  -10.7 

18.0 

1989 

25.8 

22.3 

-3.5 

28.9 

1990 

6.2 

28.1 

21.9 

17.5 

1991 

23.8 

45.1 

21.3 

16.3 

1992 

32.8 

11.4 

-21.5 

13.7 

1993 

0.0 

21.2 

21.2 

9.0 

1994 

0.0 

19.9 

19.9 

14.7 

TOTAL 

1088.8 

1134.9 

46.1 

1089.6 

Table  4-5  and  4-6  present  the  1986  ADCAP  program  SAR  budget  in  then-year 
and  base-year  figures  respectively.  Comparing  the  1986  SAR  to  the  final  SAR  indicates 
funding  is  delayed  from  the  program  and  then  returned  in  the  later  years.  A  reduction  of 
$12M  in  then-year  dollars  occurs  in  1988.  The  method  underestimates  the  final  budget 
for  this  occurrence  by  5.24%  in  then-year  dollars  and  3.99%  in  base-year  dollars.  The 
ADCAP  program  schedule  slipped  by  two  years. 


Table  4-7.  ADCAP  Program  Weibull  Parameters 


Program 

Original 

Revised 

Year  Location  Shape  Scale  t_fmal 

Year  Location  Shape  Scale  t  final 

ADCAP 

1986  |  4.8167731  0.47556|  1. 1817481  21.34731 
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Table  4-7  above  presents  the  Weibull  parameters  for  the  ADCAP  program  budget 
before  the  curtailment  (Original)  and  when  the  curtailment  is  considered  (Revised).  The 
location  parameter  remains  the  same  because  the  beginning  of  the  program  has  not 
changed.  The  change  in  the  shape  parameter  indicates  the  peak  expenditure  for  this 
program  occurs  later  than  originally  programmed.  The  scale  parameter  growth  is 
indicative  of  the  overall  program  duration.  The  t™nal  parameter  increases  by  the  noted 

schedule  growth.  The  model  for  the  ADCAP  program  gives  an  optimistic  estimate. 

The  ADDS  Program.  The  ADDS  program  is  an  Army  Electronic  program.  The 
first  SAR  report  for  this  program  was  1983  with  the  final  SAR  for  the  RDT&E  budget 
submitted  in  the  1994  SAR. 


Table  4-8.  ADDS  Program  1986  SAR  Then-Year  Budget 


Budget 

Year 

1986 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1981 

15.8 

15.8 

15.8 

1982 

17.3 

17.3 

17.3 

1983 

34.1 

34.1 

34.1 

1984 

22.9 

22.9 

22.9 

1985 

23.9 

23.9 

23.9 

1986 

33.4 

36.0 

2.6 

33.4 

1987 

35.7 

38.0 

2.3 

35.7 

1988 

33.9 

21.7 

mmmm 

21.7 

1989 

25.5 

10.5 

-15.0 

15.5 

1990 

13.0 

3.5 

-9.5 

17.1 

1991 

0.0 

3.1 

3.1 

13.9 

1992 

0.0 

6.1 

6.1 

12.1 

1993 

0.0 

7.6 

7.6 

10.4 

1994 

0.0 

14.7 

14.7 

7.9 

1995 

0.0 

2.3 

2.3 

7.9 

1996 

0.0 

1.9 

1.9 

3.0 

TOTAL 

255.5 

259.4 

3.9 

292.6 
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Table  4-9.  ADDS  Program  1986  SAR  Base-Year  Budget 


Budget 

Year 

1986 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1981 

18.6 

18.6 

18.6 

1982 

19.1 

19.1 

19.1 

1983 

35.9 

35.9 

35.9 

1984 

23.2 

23.2 

23.2" 

1985 

23.4 

23.4 

23.4 

1986 

32.0 

34.5 

2.5 

32.0 

1987 

32.7 

34.8 

2.1 

32.7 

1988 

30.2 

19.3 

— 

19.3 

1989 

21.7 

8.9 

-12.8 

13.2 

1990 

10.7 

2.9 

-7.8 

14.1 

1991 

0.0 

2.5 

2.5 

11.0 

1992 

0.0 

4.7 

4.7 

1993 

0.0 

5.8 

5.8 

7.8 

1994 

0.0 

10.9 

10.9 

5.9 

1995 

0.0 

1.7 

1.7 

5.7 

1996 

0.0 

1.4 

1.4 

2.1 

TOTAL 

247.5 

247.5 

0.0 

273.6 

Table  4-8  and  4-9  present  the  1986  SAR  report  budget  profile  in  then-year  and 
base-year  dollars  respectively  and  shows  that  funding  curtailment  occurs  in  1988  through 
1990.  The  1986  SAR  indicates  that  S12.2M  in  then-year  dollars  is  removed  from  the 
program  in  1988.  The  1986  SAR  shows  $35. 7M  in  then-year  dollars  returning  to  the 
program  starting  in  1991.  This  program  experienced  a  schedule  slip  of  6  years.  The 
method  overestimates  the  final  RDT&E  budget  by  12.80%  in  then-year  dollars  and 
10.52%  in  base-year  dollars. 


Table  4-10.  ADDS  Program  Weibull  Parameters 


Program 

Original 

Revised 

Year  Location  Shape  Scale  tfinal 

Year  Location  Shape  Scale  t  final 

ADDS 

1986  |  5.13E-09|  1.996983|  6.5709|  12.31621 

1986  |  5.13E-09I  1.536882|  8.096528|  18.31621 
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Table  4-10  presents  the  Weibull  parameters  for  the  ADDS  program  budget  before 


the  curtailment  (Original)  and  when  the  curtailment  is  considered  (Revised).  The 
location  parameter,  in  this  case  nearly  0,  indicates  the  program  receives  significant 
funding  immediately.  The  change  in  the  shape  parameter  indicates  the  peak  expenditure 
for  the  revised  budget  profile  is  proportionally  earlier  than  originally  programmed,  but 
the  program  is  longer.  The  scale  parameter  growth  is  indicative  of  the  overall  program 
duration.  The  tr™nal  parameter  increases  by  the  noted  schedule  growth — 6  years. 

The  TRI-TAC  Program.  The  TRI-TAC  program  is  an  Air  Force  Electronic 
program.  The  first  SAR  report  for  this  program  was  1983  with  the  final  SAR  for  the 
RDT&E  budget  submitted  in  the  1989  SARTable  4-11. 

Table  4-11.  TRI-TAC  Program  1983  SAR  Then-Year  Budget 


Budget 

Year 

1983 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1973 

1.8 

1.8 

1.8 

1974 

4.0 

4.0 

4.0 

1975 

7.4 

7.4 

7.4 

1976 

3.9 

3.9 

3.9 

1977 

5.0 

5.0 

■B) 

1978 

7.0 

7.0 

7.0 

1979 

4.4 

4.4 

4.4 

1980 

9.2 

9.2 

9.2 

1981 

7.4 

7.4 

7.4 

1982 

10.9 

10.9 

10.9 

1983 

25.4 

22.8 

22.8 

1984 

7.9 

7.6 

-0.3 

0.0 

1985 

4.1 

3.7 

-0.4 

10.4 

1986 

3.0 

1.0 

-2.0 

4.4 

1987 

4.4 

1.0 

7.1 

1988 

4.7 

2.7 

-2.0 

4.4 

1989 

4.8 

1.2 

-3.6 

5.0 

1990 

0.0 

3.5 

3.5 

HI 

1991 

0.0 

4.8 

4.8 

3.7 

1992 

0.0 

5.0 

5.0 

2.0 

1993 

6.0 

5.2 

5.2 

3.2 

TOTAL 

115.3 

119.5 

4.2 

127.6 
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Table  4-8  presents  the  1983  SAR  then-year  budget  figures  for  the  TRI-TAC 
program.  The  following  table  presents  the  budget  figures  for  this  program  in  base-year 
dollars. 


Table  4-12.  TRI-TAC  Program  1983  SAR  Base-Year  Budget 


Budget 

Year 

1983 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1973 

4.2 

4.2 

4.2 

1974 

8.6 

8.6 

8.6 

1975 

14.4 

14.4 

14.4 

1976 

7.0 

7.0 

7.0 

1977 

8.5 

8.5 

8.5 

1978 

11.0 

11.0 

11.0 

1979 

6.4 

6.4 

6.4 

1980 

12.0 

12.0 

12.0 

1981 

8.7 

8.7 

8.7 

1982 

12.0 

12.0 

12.0 

1983 

26.7 

24.0 

mmm 

24.0 

1984 

8.0 

7.7 

-6.3 

0.0 

1985 

4.0 

3.6 

-0.4 

10.2 

1986 

2.9 

1.0 

-1.9 

4.3 

1987 

4.0 

0.9 

-3.1 

6.5 

1988 

4.2 

2.4 

-1.8 

3.9 

1989 

4.1 

1.0 

-3.1 

4.3 

1990 

0.0 

2.9 

2.9 

2.8 

1991 

0.0 

3.8 

3.8 

2.9 

1992 

0.0 

3.9 

3.9 

1.6 

1993 

0.0 

3.9 

3.9 

2.4 

TOTAL 

146.6 

147.8 

1.2 

155.6 

Table  4-12  presents  the  1983  SAR  for  the  TRI-TAC  program  in  base-year  dollars. 
Both  tables  show  funding  curtailment.  Funding  is  reduced  is  in  1983.  During  this  year, 
the  program  experiences  four  years  of  schedule  slip.  The  method  overestimates  the  final 
RDT&E  budget  by  6.74%  in  then-year  dollars  and  5.28%  in  base  year  (constant)  dollars. 
For  this  program,  the  proposed  method  estimates  a  larger  (conservative)  budget. 
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Table  4-13.  TRI-TAC  Program  Weibull  Parameters 


Program 

Original 

Revised 

Year  Location  Shape  Scale  tfmal 

Year  Location  Shape  Scale  tfmal 

TRI-TAC 

1983  |  1.1451651  2.234193|  10.79748|  20.07753 

1983  |  1.145165|  1.9822291  12.177721  24.07753 

Table  4-13  above  presents  the  Weibull  parameters  for  the  TRI-TAC  program 
budget  before  the  curtailment  (Original)  and  when  the  curtailment  is  considered 
(Revised).  The  location  parameter  remains  the  same  because  the  beginning  of  the 
program  has  not  changed.  The  change  in  the  shape  parameter  indicates  the  peak 
expenditure  for  this  budget  profile  is  proportionally  earlier  than  originally  programmed, 
but  the  program  is  much  longer.  As  with  previous  modeled  programs,  the  scale 
parameter  increase  indicates  schedule  growth  of  the  program.  The  t™nal  parameter 

increases  by  the  noted  four-year  schedule  growth. 

The  WAM  Program.  The  WAM  Program  is  an  Air  Force  Electronic  program. 
This  program  receives  funding  from  all  military  services  plus  other  defense  agencies. 
The  first  SAR  report  for  this  program  was  in  1983  with  the  final  SAR  for  the  RDT&E 
budget  submitted  in  the  1991  SAR. 

This  program  shows  funding  curtailment  in  three  years:  1983,  1989,  and  1990. 
WAM  program  requirements  were  not  finalized  until  1985.  Since  we  intend  to  model 
program  with  constant  program  requirements,  we  exclude  the  funding  curtailment  noted 
in  1983. 


4-13 


Table  4-14.  WAM  Program  1989  SAR  Then-Year  Budget 


Budget 

Year 

1989 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1982 

14.0 

14.0 

14.0 

1983 

17.8 

17.8 

17.8 

1984 

46.5 

46.5 

46.5 

1985 

76.0 

76.0 

76.0 

1986 

104.5 

104.5 

104.5 

1987 

138.4 

138.4 

138.4 

1988 

59.6 

59.1 

-0.5 

59.6 

1989 

96.4 

96.3 

-0.1 

96.4 

1990 

81.8 

75.0 

-6.8 

81.8 

1991 

83.3 

73.6 

-9.7 

83.3 

1992 

96.6 

59.1 

111  -37J5 

59.1 

1993 

79.5 

44.1 

-35.4 

74.9 

1994 

70.5 

51.3 

-19.2 

52.8 

1995 

0.0 

49.8 

49.8 

43.1 

1996 

0.0 

44.4 

44.4 

45.7 

1997 

0.0 

29.6 

29.6 

10.3 

TOTAL 

964.9 

979.5 

14.6 

1004.2 

Table  4-15.  WAM  Program  1989  Base-Year  Budget 


Budget 

Year 

1989 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1982 

15.4 

15.4 

15.4 

1983 

18.7 

18.7 

18.7 

1984 

47.1 

47.1 

47.1 

1985 

74.5 

74.5 

74.5 

1986 

100.0 

100.0 

100.0 

1987 

126.7 

126.7 

126.7 

1988 

53.1 

52.7 

-0.4 

53.1 

1989 

82.0 

81.9 

-0.1 

82.0 

1990 

67.5 

61.9 

-5.6 

67.5 

1991 

66.3 

58.5 

-7.7 

66.3 

1992 

74.7 

45.7 

45.7 

1993 

60.2 

33.4 

-26.8 

56.7 

1994 

52.5 

38.2 

-14.3 

39.3 

1995 

0.0 

36.4 

36.4 

31.4 

1996 

0.0 

31.9 

31.9 

32.8 

1997 

0.0 

21.0 

21.0 

7.3 

TOTAL 

838.7 

844.0 

5.3 

864.7 
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Table  4-14  and  4-15  present  the  1989  SAR  and  shows  $92. 1M  in  then-year 
dollars  being  curtailed  from  the  program  starting  in  1992  and  $123.8M  in  then-year 
dollars  reprogrammed  into  the  program’s  later  years.  The  WAM  program  experiences 
three  years  of  schedule  slip  in  the  1989  SAR.  The  method  overestimates  the  final 
RDT&E  budget  by  2.53%  in  then-year  dollars  and  2.45%  in  base-year  dollars. 


Table  4-16.  WAM  Program  1990  SAR  Then-Year  Budget 


Budget 

Year 

1990 

Budget 

Final  SAR 
Budget 

Difference 

Revised 

Budget 

1982 

14.0 

14.0 

14.0 

1983 

17.8 

17.8 

17.8 

1984 

46.5 

46.5 

46.5 

1985 

76.0 

76.0 

76.0 

1986 

104.5 

104.5 

104.5 

1987 

138.4 

138.4 

138.4 

1988 

59.1 

59.1 

59.1 

1989 

96.3 

96.3 

96.3 

1990 

75.0 

75.0 

75.0 

1991 

83.2 

73.6 

-9.6 

83.2 

1992 

69.7 

59.1 

-10.6 

69.7 

1993 

63.1 

44.1 

lllillil! 

44.1 

1994 

67.8 

51.3 

-16.5 

56.1 

1995 

0.0 

49.8 

49.8 

30.1 

1996 

0.0 

44.4 

44.4 

41.7 

1997 

0.0 

29.6 

29.6 

7.3 

TOTAL 

911.4 

979.5 

68.1 

959.8 

4-15 


Difference 


Table  4-17.  WAM  Program  1990  SAR  Base-Year  Budget 


Revised 

Budget 


1982  I  15.41  15.4] _ I  15.4 


1983  |  18.7  18.7  I  18.7 


1984  T  47.1  47.1  I  47.1 


1985  |  7A5]  74.5  I  74.5 


1986  100.0  100td _ I  100.0 


1987  126.7  126.7  126.7 


1988  |  52.7  52.71 _ I  52.7 


1989  81.9  81.9  81.9 


1990  T  61.9  61.9  61.9 


1991  66.2  58.5  -7.6  66.2 


1992  |  53.9  45.7|  -8.2|  53.9 


1993  47.8  33.41  -14.4|  33.4 


1994  50.5  38.2  -12.3|  41.8 


1995  0.0  36.4  36.4 1  22.0 


1996  0.0  31.9  31.91  29.9 


1997  0.0  21.0  21.0  5 .2 


TOTAL  797.3  844.01  46.71  831.3 


The  1990  SAR  budgets  displayed  in  Table  4-16  (then-year  dollars)  and  Table  4- 
17  (base-year  dollars)  shows  S35.5M  in  then-year  dollars  curtailed  from  the  program 
starting  in  1993  and  $123. 8M  in  then-year  dollars  being  reprogrammed  into  the 
program’s  out  years.  The  1990  SAR  indicates  the  program  slips  three  years  when  this 
curtailment  occurs.  The  method  underestimates  the  final  budget  by  2.01%  in  then-year 
dollars  and  1.50%  in  base-year  dollars.  The  model  underestimates  the  final  RDT&E 
budget  in  both  applications. 


Table  4-18.  WAM  Program  Weibull  Parameters 


Original 

Revised 

Program 

Year 

Location 

Shape 

Scale 

t  final 

Year 

Location 

Shape 

Scale 

t  final 

WAM  Summary 

1989 

0.711592 

2.062697 

8.188366 

15.75536 

1989 

0.711592 

1.91584 

9.373868 

18.75536 

WAM  Summary 

1990 

1.204005 

1.874529 

7.421058 

15.69646 

1990 

1.204005 

1.721063 

8.438306 

18.69646 
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Table  4-18  above  presents  the  Weibull  parameters  for  the  WAM  program  budget 


before  the  curtailment  (Original)  and  when  the  curtailment  occurred  (Revised).  The 
location  parameter  remains  the  same  because  the  beginning  of  the  program  has  not 
changed.  The  change  in  the  shape  parameter  indicates  the  peak  expenditure  for  this 
budget  profile  is  proportionally  earlier  than  originally  programmed.  The  scale  parameter 
growth  is  indicative  of  the  overall  schedule  growth  of  the  program.  The  tr^naI  parameter 
increases  by  the  schedule  growth. 


Results 

The  methodology  outlined  in  Chapter  Three  is  applied  to  the  programs  meeting 
the  criteria  put  forth  at  the  beginning  of  this  chapter.  The  final  budget  forecast  using  the 
methodology  is  compared  with  the  final  SAR  containing  RDT&E  expenditures.  All 
comparisons  are  done  in  then-year  dollars  and  base-year  dollars  to  assure  the  same 
baseline  is  being  used. 


Table  4-19.  Method  Accuracy  Summery 


Program 

Year 

Method  Accuracv 

Then- Year 

1985 

-1.25% 

-0.23% 

ADCAP 

1986 

-5.24% 

-3.99% 

ADDS 

1986 

12.80% 

10.52% 

TRI-TAC 

1983 

6.74% 

5.28% 

WAM 

1989 

2.53% 

2.45% 

WAM 

1990 

-2.01% 

-1.50% 

Average  Method  Accuracy 

2.26% 

2.09% 

Average  Absolute  Value 

5.10% 

4.00% 

Table  4-19  presents  all  the  programs  and  the  method  accuracy  for  each  respective 
program.  The  first  column  is  the  program  name.  The  second  column  is  the  year  where 


funding  was  curtailed.  The  third  column  is  the  method’s  accuracy  to  predict  the  final 
budget  as  a  percentage  of  the  actual  final  RDT&E  budget  SAR  in  then-year  dollars.  The 
fourth  column  is  the  same  comparison  as  the  third  column  but  base-year  (constant) 
dollars  are  used.  The  bottom  row  is  the  average  method  accuracy  error.  Again,  the 
closer  the  percentage  is  to  zero,  the  better  the  method’s  accuracy  to  predict  the  final 
budget. 

Discussion  of  Results.  The  model  forecasts  nearly  all  of  the  programs  within  +/- 
7%  of  the  final  RDT&E  budget  SAR.  The  model  forecasts  all  program’s  final  RDT&E 
budget  SAR  within  +/- 13%. 

The  method’s  average  error  is  2.26%  over  the  actual  then-year  dollar  budget. 

This  indicates  the  model  is  a  little  conservative  in  its  estimation  of  the  final  budget.  The 
method’s  average  error  improves  to  an  overestimation  of  2.09%  in  base-year  dollars.  The 
low  percentage  of  error  is  considered  good  and  supports  the  conclusion  that  this 
methodology  provides  a  good  approximation  of  the  effect  of  a  budget  curtailment. 
Generally,  the  estimated  budget  was  higher  than  the  actual  budget  for  the  year  following 
the  curtailment.  Since  authority  for  the  curtailed  funds  is  usually  transferred  from  the 
military  department  to  the  Office  of  the  Secretary  of  Defense  (OSD),  we  are  not  surprised 
that  programs  that  are  curtailed  are  not  fully  funded  in  the  next  fiscal  year.  Why  should 
the  military  departments  submit  money  that  has  a  high  risk  of  being  taken?  As  a  result, 
most  programs  that  are  curtailed  have  several  following  years  where  the  base-year  dollar 
total  of  their  program  is  below  the  constant  dollar  amount  required  to  complete  the 
program.  The  underfunding  of  R&D  programs  made  us  use  the  final  SAR,  which  by 
definition  is  a  fully  funded  program,  as  the  requirement.  In  base-year  dollars,  our  method 
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is  even  more  conservative.  In  conclusion,  the  result  of  this  approach  to  forecast  budgets 
for  programs  in  the  RDT&E  phase  when  funding  curtailment  is  considered  successful. 

Chapter  Summary 

This  chapter  provides  the  results  of  applying  the  methodology  to  historical 
program  costs  data.  A  brief  description  of  each  program  meeting  the  criteria  was  given 
along  with  information  about  the  funding  curtailment  observed.  The  percentage  of  error 
provides  model  forecasting  accuracy  of  the  final  RDT&E  budget.  Finally,  model 
accuracy  for  all  programs  is  shown  with  discussion  of  those  results. 

Chapter  Three  outlines  the  proposed  methodology.  We  define  success  as  being 
able  to  have  the  model  forecast  what  the  final  RDT&E  budget  would  be.  This  chapter 
reports  excellent  results  from  six  applications.  The  following  chapter,  Chapter  Five, 
discusses  the  conclusions  that  are  drawn  based  upon  the  analysis  performed  in  this 
chapter. 
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V.  Conclusion  and  Recommendations 


Chapter  Overview 

Summary  findings  and  conclusions  of  this  research  effort  are  presented  in  this 
chapter.  First,  a  brief  restatement  of  the  findings  from  Chapters  One,  Two,  and  Three, 
and  summary  of  the  results  from  Chapter  Four  are  provided.  Next,  general  conclusions 
regarding  this  overall  research  effort  are  discussed.  This  discussion  includes  a 
comparison  of  the  results  of  this  research  effort  to  previous  efforts.  Finally, 
recommendations  of  areas  for  future  related  research  are  presented 

Synopsis 

Chapter  One  introduces  the  problem  of  calculating  the  consequences  of  funding 
curtailment  to  a  program  in  the  R&D  phase.  In  addition,  it  brings  to  light  that  the 
consequences  most  often  associated  with  funding  curtailment  are  program  cost  and 
schedule  growth.  Finally,  the  chapter  concludes  that  the  purpose  of  this  research  is  to 
provide  a  quick  and  reasonably  accurate  estimation  tool  of  funding  impact  when 
considering  funding  curtailment. 

Chapter  Two  summarizes  the  findings  of  the  literature  review  for  this  research 
effort.  We  discuss  several  cost  models  before  explaining  the  capabilities  of  the  Rayleigh 
model.  Several  cited  papers  show  that  the  Rayleigh  model  has  the  capability  to  forecast 
project  budgets.  We  identify  the  parameters  needed  to  use  the  Rayleigh  model  and  we 
provide  information  on  how  each  parameter  is  derived.  We  describe  the  Weibull  function 
as  the  means  to  model  real  program  expenditures.  The  flexibility  of  the  Weibull  function 
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provides  a  more  accurate  budget  and  expenditure  model.  The  chapter  concludes  with  a 
discussion  of  the  impact  outlay  rates  have  on  the  budget  profile. 

Chapter  Three  outlines  the  methodology  we  use  to  develop  budget  profiles  when 
considering  funding  curtailment.  We  explain  the  steps  for  converting  then-year  dollars  to 
base-year  dollars.  The  chapter  outlines  the  nonlinear  programming  step  we  take  to  derive 
the  Weibull  parameters  for  the  original  and  revised  budgets.  Finally,  we  explain  how  the 
revised  budget  is  built  using  past  years’  budgets  for  years  already  past,  the  planned 
budget  amount  for  the  year  of  the  curtailment,  and  the  estimated  budget  based  on  Weibull 
forecast  amounts  for  future  years. 

Chapter  Four  provides  the  results  of  the  methodology  applied  to  real  historical 
programs.  We  explain  the  criteria  used  in  selecting  the  programs  along  with  the  rationale 
for  each  criterion.  The  tables  show  the  curtailment  of  each  selected  program.  We 
explain  the  Weibull  parameters  of  the  selected  programs.  We  present  the  definition  of 
success  as  the  ability  of  the  methodology  to  forecast  the  same  amount  as  the  final 
RDT&E  SAR  budget  amount.  All  selected  program  budget  forecasts  were  within  6%  of 
the  actual  budget  amount.  The  results  are  presented  as  a  percentage  of  how  accurately 
the  method  is  able  to  forecast  the  final  budget  amount. 

Conclusion 

We  conclude  the  proposed  methodology  does  provide  a  very  accurate  and  quick 
ability  to  forecast  revised  R&D  budgets  when  funds  are  delayed.  Thus,  we  achieved  our 
research  objective.  The  method  forecasts  nearly  all  of  the  programs  within  +1-1%  of  the 
final  RDT&E  budget  SAR  with  an  average  method  accuracy  error  of  2.26%  in  then-year 


5-2 


dollars  and  2.09%  in  base-year  dollars.  This  indicates  the  model  is  a  little  conservative  in 
its  estimation  of  the  final  budget.  We  consider  the  low  percentage  of  error  good,  which 
supports  the  conclusion  that  this  methodology  provides  a  good  approximation  of  the 
effect  of  a  budget  curtailment. 

Areas  for  Future  Related  Research 

Contributing  Factors.  During  the  analysis  of  the  modeled  programs,  we  tracked 
several  characteristics  of  the  program.  For  example,  for  each  modeled  program  we  noted 
the  percent  of  the  curtailment  when  compared  to  the  overall  budget,  how  far  along  the 
program  was  before  the  curtailment  occurred,  and  the  Weibull  parameters  for  each 
program  among  other  characteristics. 

Because  of  the  small  number  of  programs  modeled  in  establishing  this  approach, 
insufficient  data  exists  to  statistically  distinguish  factors  that  can  be  identified  as 
contributing  to  or  derogating  from  the  model’s  accuracy.  Future  research  could  be 
conducted  in  search  of  identifiable  factors  that  would  provide  the  reader,  and  ultimately 
the  user,  a  better  picture  of  what  contributes  to  the  model’s  accuracy  or  inaccuracy  as  the 
case  may  be. 

Other  Cost  Models.  This  research  effort  uses  the  Rayleigh  Model  and  its  parent, 
the  Weibull  Model,  as  the  means  to  model  each  program.  Chapter  Two  highlights  other 
cost  models  available  in  the  literature.  Although  we  chose  not  to  use  these  other  models 
for  reasons  stated,  there  is  value  in  applying  those  cost  models  to  learn  the  accuracy  of 
providing  an  accurate  forecast  of  future  budgets  when  considering  funding  curtailment. 
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Appendix  A.  An  Alternative  Approach 


Appendix  Note 

There  are  different  methods  available  to  address  the  research  question.  During 
the  research  effort,  we  pursued  a  path  we  thought  would  provide  the  analytic 
methodology  we  sought.  After  several  weeks  of  research  into  this  area,  we  presented  this 
methodology  to  Dr.  Burke  (OSD  PA&E),  our  sponsor.  He  informed  us  that  one  of  the 
assumptions  we  were  using  was  incorrect.  Work  accomplished  does  not  provide  benefit 
to  the  project  if  resources  are  not  available  to  use  the  work  results.  Needless  to  say,  we 
pursued  another  methodology. 

We  have  included  this  alternative  methodology  to  aid  the  reader  in  understanding 
why  this  approach,  at  this  time,  would  not  be  appropriate.  The  mathematical  basis  for  the 
methodology  is  sound  and  is  provided. 

Appendix  Overview 

This  appendix  follows  the  reasoning  laid  out  in  Chapter  Two  as  the  basis  for  using 
the  Rayleigh  function  to  model  expenditure  profiles.  Figure  2-1  presents  a  cumulative 
expenditure  profile  for  a  program.  Figure  3-1  presents  cumulative  expenditure  profiles 
for  a  program  with  a  funding  curtailment.  The  parameters  needed  for  this  methodology 
are  the  same  as  presented  in  Chapter  Two. 

The  methodology  diverges  from  the  methodology  in  Chapter  Three  at  the  point  of 
curtailment. 
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Once  the  final  time  has  been  determined,  the  next  step  is  to  derive  an  estimate  of 
the  funding  profile  needed  that  will  follow  the  Rayleigh  probability  density  function.  The 
starting  point  is  the  period  where  actual  time  and  expenditures  have  occurred  and  ends  at 
the  revised  final  time. 

The  Rayleigh  model  has  been  presented  as  a  function  of  time.  The  model  states 
how  much  money  is  spent  in  the  Rayleigh  cumulative  expenditure  function  distribution 
and  the  rate  of  expenditures  in  the  Rayleigh  probability  expenditure  distribution  at  a 
given  time.  Gallagher  and  Lee  provide  key  insight  on  how  the  research  question  can  be 
answered  (Gallagher,  1996:52-53).  In  order  to  solve  the  thesis  question,  the  focus  needs 
to  change  to  look  at  the  rate  of  expenditure  at  a  specific  earned  value — amount  spent  on  a 
project.  In  essence,  at  what  rate  should  the  project  be  spending  now  that  the  project  is 
has  spent  a  specific  amount? 

We  know  from  the  Rayleigh  model  literature  that  during  the  first  phase  of  the 
project  the  rate  of  spending  increases  up  to  a  certain  peak  point,  tpeak.  After  that  point  is 
reached,  the  rate  of  spending  decreases  as  the  proportion  of  problems  remaining  to  be 
solved  decreases. 

This  expenditure  rate  is  assumed  to  be  continuous,  which  means  that  the 
expenditure  rate  the  project  had  one  day  will  be  similar  to  the  expenditure  rate  the  next 
day.  The  expenditure  rate  increases  prior  to  its  peak  and  decreases  after  the  peak  is 
attained.  In  other  words,  the  expenditure  rate  is  not  disjointed.  The  expenditure  rate  will 
not  be  drastically  different  in  contiguous  time  periods.  This  is  evidenced  by  the  smooth 
line/curve  of  the  Rayleigh  distributions. 
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Up  to  this  point,  we  understand  that  the  expenditure  profile  follows  the  Rayleigh 
function.  We  also  know  that  if  funding  is  changed,  that  a  new  Rayleigh  function  can  be 
applied  to  model  this  new  change.  This  new  Rayleigh  function  will  project  a  new  project 
completion  time,  t™ml .  Let  v  equal  the  amount  of  earned  value — the  amount  already 

spent  on  the  project.  The  derivative  of  v  with  respect  to  time  provides  the  expenditures  at 
certain  point  in  time,  dv/dt.  The  change  in  funding  usually  occurs  after  an  expenditure  of 
funds  has  already  occurred,  at  some  point  in  time,  tv 

We  present  dv/dt  as  a  function  of  time.  If  we  derive  dv/dt  as  a  function  of 
cumulative  expenditures  (earned  value),  v,  then  the  appropriate  rate  of  expenditure  can  be 
projected  when  the  amount  of  earned  value  is  known. 

The  task  is  to  fit  a  Rayleigh  model  that  starts  at  point  (£,  Vi)  and  terminates  at 
point  ,  d) — the  same  final  point  of  the  new  Rayleigh  function.  This  task  is 
illustrated  in  Figure  A-l  and  the  following  simple  example  scenario. 


Figure  A-l,  Conditional  Rayleigh  Incorporating  Time  and  Expenditures 
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For  the  example  scenario,  assume  Figure  A-l  is  the  graphical  representative 
model  of  a  current  program.  The  solid  line  represents  the  original  cumulative 
expenditures  of  the  sample  program.  This  line  follows  the  Rayleigh  cumulative 
distribution.  At  Time  3,  the  mangers  wish  to  reduce  the  rate  at  which  resources  are 
consumed  by  this  program.  Therefore,  the  completion  of  the  program  is  delayed  from 
torg  to  trev 

1 final  lU  1  final- 

The  new  completion  point  using  the  Rayleigh  model  will  have  a  different  shaping 
parameter  a  than  the  original  Rayleigh  model.  If  the  program  had  started  with  this  new 
funding  profile,  assume  the  Rayleigh  cumulative  density  function  would  have  been 
similar  to  the  dashed  ‘Hypothetical  Rayleigh’  line  in  Figure  A-l. 

Since  funds  have  been  expended  to  point  v,  and  three  time  periods  have  passed,  4 
the  ‘Hypothetical  Rayleigh’  line  does  not  take  into  consideration  the  time  and  money 
actually  used.  Therefore,  a  new  equation  based  on  the  Rayleigh  model  must  be 
developed  that  takes  into  consideration  the  money  and  time  already  put  forth  into  the 
program’s  R&D  effort.  The  equation  should  start  at  point  (4  v,)  and  end  at  t™nal .  The 

dotted  line  ‘Conditional  Rayleigh’  in  Figure  A-l  represents  the  equation  of  that  line. 

Up  to  this  point,  the  goal  of  calculating  the  Condition  Rayleigh  provides  us  with 
our  objective.  Gallagher  and  Lee  provide  adaptable  mathematical  formulation  techniques 
that  will  suite  our  purpose.  It  is  assumed  the  reader  is  knowledgeable  of  basic  algebra 
and  calculus.  For  an  in  depth  analysis  of  the  equation,  the  reader  is  referred  to  Gallagher, 
1996.  The  next  step  is  to  derive  an  equation  that  will  allow  us  to  ‘draw’  a  Rayleigh  curve 
from  point  (4  v,)  and  end  at  t™nal . 
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Let  v  equal  the  amount  of  earned  value  in  constant  dollars — the  amount  already 
spent  on  the  project,  the  derivative  of  v  with  respect  to  time  will  provide  the  needed 
expenditure  rate.  Funding  cannot  be  negative — once  a  program  spends  it,  it  cannot  be 
recovered.  As  such,  the  derivative  of  any  function  of  v  will  always  be  positive. 

Assume  the  equation  F(v)  needs  to  be  stated  in  how  much  spent  per  unit  of  time. 
As  stated  before,  let  v  equal  the  amount  of  earned  value — the  amount  already  spent  on  the 
project,  the  derivative  of  v  with  respect  to  time  provides  the  needed  expenditure  rate. 

This  can  be  shown  as 

(,8) 

Assume  that  a  function,  in  this  case  P(v)  can  be  related  to  time.  In  other  words, 
given  a  certain  time  point,  the  result  from  equation  P( v)  can  be  derived.  In  equation, 

P(v)  =  t.  (19) 

To  relate  (18)  with  (19),  (19)  must  be  solved  for  dv/dt.  The  derivative  of  (19)  with 
respect  to  time  would  be 

P'(v)dv  =  dt. 

Divide  both  sides  by  dt 

P'(v)dv  _ 
dt 


Solving  for  dv/dt  gives 

dv  _  1 

~dt~~P'  (v) 

By  substituting  (18)  into  (20),  we  derive 


(20) 
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(21) 


F(v)  =  —  =  — 

v  7  M  DW 


dt  F(v) 

through  algebraic  manipulation  the  above  equation  can  be  stated 

1 


F(v)  = 


F(v) 


F(v)  is  positive  because  the  rate  of  expenditures,  dv/dt,  cannot  be  negative.  Therefore, 

P  ’(v)  is  always  positive,  and  P(v)  increases  continually.  This  means  that  for  each  point  of 
cumulative  expenditures  a  single  corresponding  time  exists.  In  other  words,  for  each 
earned  value  point  there  is  exactly  one  corresponding  point  in  time.  This  characteristic 
ensures  the  inverse  of  the  function  P( v)  exists.  Stated  in  a  formula 

P~'(t)  =  v. 

These  above  rules  hold  for  any  function.  The  purpose  of  our  thesis  question  concerns  the 
Rayleigh  function.  To  put  the  Rayleigh  function  in  the  context  described  here,  the 
Rayleigh  function  can  be  described  as  (adapted  from  (3)) 

v =4 -(,-■)) 

We  solve  for  t  to  get  P(v)  by  first  dividing  out  the  d  parameter  resulting  in 


v  -at2 

-  =  l-e  . 
d 


Bringing  e  to  the  left  side  of  the  equation  to  make  it  positive  results  in 


e~*  =1 


Taking  the  natural  logarithm  to  remove  e  results  in 

-at2=  ln(l-j) 

Dividing  through  by  a  results  in 
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t'=—  ln(l-f) 
a 

Taking  the  square  root  of  both  sides  to  remove  the  exponential  of  the  t  parameter  results 


in 


'  =  (->(!-#•  (22) 

From  (22),  we  now  have  P(v)  for  the  Rayleigh  function  (3).  Equation  (7)  provides  the 
mathematical  relation  ship  between  P(v)  and  F(v).  Therefore,  to  get  F(v)  from  the  P(v) 
the  derivative  of  P(v)  (22)  must  be  taken.  The  derivative  is 

<*  =  i  [-  i  ln(l  -  (-  i)-^— (- 
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Combining  like  terms  results  in 


(  i  A 


dt  = 


lad 


1  _  V. 

V1  dj 


[-7ln(l-#^v. 


By  inverting  the  preceding  equation,  we  will  obtain  dv/dt  which  per  (18)  equals  F(v) 

F(v)  =  ^=2arf(l-jX->(l-7)li. 

As  stated  before,  all  expenditures  are  positive.  Only  at  the  inception  of  the  program  are 
they  zero.  Stated  in  formula  notation  is 

v(0)  =  0. 

At  any  point  in  time,  tu  the  expenditure  rate  will  be  v,, 

vft)  =  v,. 

Solving  (18)  with  initial  conditions  of  v(tj  =  v,  we  arrive  at 

v(0  =  P-'(/-/,)+P(v,) 
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Substituting  this  back  into  the  Rayleigh  function  results  in 


By  substituting  in  the  formula  of  P(v)  above,  the  final  formula  is 


f 


v(t)  =  d 


l-e 


(23) 


V  J 

Now  that  we  have  the  needed  equation,  the  final  task  is  to  use  (23)  with  the 
relevant  time  and  earned  value  numbers  to  model  R&D  programs  that  have  had  changes 
to  their  funding  and/or  expenditure  profiles.  The  final  step  to  accomplish  this  is  to 
extrapolate  the  budget  profile  through  the  outlay  and  appropriation  filters  from  a  required 
outlay  profile. 


Appendix  Summary 

In  summary,  the  methodology  we  propose  can  be  broken  down  into  the  following 
steps:  A  new  final  time  is  determined,  tr™naX .  This  final  time  provides  the  appropriate  a 

parameter  to  use  in  (23).  An  outlay  profile  will  be  developed  using  (23)  and  from  this 
outlay  profile,  the  same  approach  may  be  applied  to  determine  the  budget. 
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Appendix  B.  Inflation  Indices  for  Model  Programs 


Appendix  Overview 

Converting  then-year  dollars  to  base-year  dollar  requires  inflation  factors.  This 
appendix  contains  the  raw  inflation  indices  for  the  selected  programs.  We  use  these 
factors  to  remove  inflation  from  yearly  expenditures.  Bolded  numbers  are  the  base  year 
for  the  particular  program. 

The  final  table  contains  weighted  inflation  factors.  We  use  these  weighted  factors 
when  removing  inflation  from  a  budget. 
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Program  Inflation  Indices 

MLRS-TGW  Program.  The  MLRS-TGW  program  is  an  Army  Missile  program 
The  base  year  for  this  program  is  1984. 


Table  B-l.  Inflation  Index  for  the  MLRS-TGW  Program 


The  ADCAP  Program.  The  ADCAP  program  is  a  Navy  Munitions  program.  The 


base  year  for  this  program  is  1989. 


Table  B-2.  Inflation  Index  for  the  ADCAP  Program 


Year 

Index 

1979 

0.586 

1980 

0.641 

1981 

0.718 

1982 

0.784 

1983 

0.822 

1984 

0.854 

1985 

0.883 

1986 

0.907 

1987 

0.932 

1988 

0.960 

1989 

1.000 

1990 

1 .040 

1991 

1.085 

1992 

1.115 

1993 

1.145 

1994 

1.168 

1995 

1.190 

1996 

1.214 

1997 

1.240 

1998 

1.248 

1999 

1.258 

2000 

1.271 

2001 

1.290 

2002 

1.309 

2003 

1.329 

2004 

1.355 

2005 

1.383 

2006 

1.410 

2007 

1.438 

2008 

1.467 

2009 

1.497 
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The  ADDS  Program.  The  ADDS  program  is  an  Army  Electronic  program.  The 


base  year  for  this  program  is  1983. 


Table  B-3.  Inflation  Index  for  the  ADDS  Program 


Year 

Index 

1981 

0.873 

1982 

0.953" 

1983 

1.000 

1984 

1.038 

1985 

1.073 

1986 

1.103 

1987 

1.133 

1988 

1.167 

1989 

1.216 

1990 

1.265 

1991 

1.319 

1992 

1.356 

1993 

1 .393 

1994 

1.421 

1995 

1.448 

1996 

1.477 

1997 

1.508 

1998 

1.518 

1999 

1.530 

2000 

1.546 

2001 

1.569 

2002 

1.592 

2003 

1.616 

2004 

1.648 

2005 

1.681 

2006 

1.715 

2007 

1.749 

2008 

1.784 

2009 

1.820 

2010 

1.856 

2011 

1.894 
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The  TRI-TAC  Program.  The  TRI-TAC  program  is  an  Air  Force  Electronic 


program.  The  base  year  for  this  program  is  1991. 


Table  B-4.  Inflation  Index  for  the  TRI-TAC  Program 


Year 

Index 

1992 

1.028 

1993 

1.056 

1994 

1.077 

1995 

1.097 

1996 

1.119 

1997 

1.143 

1998 

1.151 

1999 

1.160 

2000 

1.172 

2001 

1.189 

2002 

1.207 

2003 

1.225 

2004 

1.250 

2005 

1.275 

2006 

1.300 

2007 

1.326 

2008 

1.353 

2009 

1.380 

2010 

1.407 

2011 

1.435 

2012 

1.464 

2013 

1.493 

2014 

1.523 

2015 

1.554 

2016 

1.585 

2017 

1.617 

2018 

1.649 

2019 

1.682 

2020 

1.715 

2021 

1.750 

2022 

1.785 
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The  WAM  Program.  The  WAM  Program  is  an  Air  Force  Electronic  program. 


The  base  year  for  this  program  is  1982. 


Table  B-5.  Inflation  Index  for  the  WAM  Program 


Year 

Index 

1982 

1.000 

1983 

1.049 

1984 

1.089 

1985 

1.126 

1986 

1.157 

1987 

1.189 

1988 

1.224 

1989 

1.276 

1990 

1.327 

1991 

1.384 

1992 

1.423 

1993 

1.461 

1994 

1.490 

1995 

1.519 

1996 

1.549 

1997 

1.581 

1998 

1.592 

1999 

1.605 

2000 

1.621 

2001 

1.646 

2002 

1.670 

2003 

1.695 

2004 

1.729 

2005 

1.764 

2006 

1.799 

2007 

1.835 

2008 

1.872 

2009 

1.909 

2010 

1.947 

2011 

1.986 

2012 

2.026 
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Weighted  Inflation  Factors 

The  weighted  inflation  factors  are  used  to  remove  inflation  from  budget  figures. 
This  table  is  used  for  the  base-year  comparisons. 


Table  B-6.  Weighted  Inflation  Index  for  Budgets 
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